Mnangagwa, until recently one of Mugabe's closest allies, took the oath of office at the national sports stadium on the outskirts of Harare before thousands of supporters, dignitaries and foreign diplomats.
Snipers took up positions around the stadium amid tight security as jubilant Mnangagwa supporters streamed in, many dancing as music played.
The man known as "The Crocodile" wore a dark suit, red tie and matching handkerchief, and a row of medals on his chest for the ceremony.
"Mugabe is very old -- we do not expect that he will be punished for his crimes. How do you punish a 93-year-old? But his wife and others must be charged if they committed crimes."
More From This Section
The former president, who ruled the southern African country for 37 autocratic years, was ousted from office when the military intervened after he had sacked Mnangagwa as vice president.
"We thank you our soldier," said one banner in the sports ground. "The people have spoken," said another.
Mugabe is in increasingly frail health and had been positioning his wife Grace as his successor, but the army chiefs acted to halt the plan and usher in Mnangagwa.
State-run media had earlier claimed that Mugabe may even attend his successor's swearing-in -- but later suggested that after he and Mnangagwa talked about the inauguration, he agreed he "needed time to rest".
Mnangagwa also "assured him and his family maximum security and welfare" for their future as private citizens, the state-run Herald news site also reported.
Buses brought well-wishers to the 60,000-capacity stadium from the early hours today.
Leader of the main opposition Movement for Democratic Change Morgan Tsvangirai received rapturous applause as he arrived at the packed stadium for the swearing-in.
Opposition stalwart Joice Mujuru was also at the event at which military units stood in formation on a multi-purpose grass sports pitch, under the shadow of a big-screen TV.