The talks in Harare, confirmed to AFP by a source close to the army leadership, came after soldiers blockaded key roads, took over state TV and detained the veteran leader.
"They met today. He is refusing to step down. I think he is trying to buy time," said the source, who declined to be named.
Mugabe's motorcade reportedly took him from his private residence to State House for talks which were held alongside meetings with envoys from the Southern African Development Community (SADC) regional bloc.
Zimbabwe was left stunned at the military intervention against Mugabe, 93, who has ruled the country since independence from British rule in 1980.
More From This Section
Despite Mugabe's refusal to resign, attention has shifted to the prominent figures who could play a role in any transitional government.
Morgan Tsvangirai, a former prime minister and long-time opponent of Mugabe, told journalists in Harare that Mugabe must resign "in the interest of the people".
Tendai Biti, an internationally-respected figure who served as finance minister during the coalition government after the 2008 elections, called it "a very delicate time for Zimbabwe".
"A way has to be worked out to maintain stability. That restoration requires a roadmap and to address the grievances that have led to this situation," he said.
Mugabe's advanced age, poor health and listless public performances fuelled the bitter succession battle between his wife Grace and vice president Emmerson Mnangagwa, who Mugabe sacked last week.
Mnangagwa, 75, was previously one of Mugabe's most loyal lieutenants, having worked alongside him for decades.
The military generals were strongly opposed to Grace's rise, while Mnangagwa has maintained close ties to the army and could emerge as the next president.
"People want the constitution to be upheld. The talks should look at how to deal with the Mugabe issue in a progressive manner," political analyst Earnest Mudzengi told AFP.
Eldred Masunungure, a lecturer at the University of Zimbabwe, added that the formation of a "pre-election coalition" could be a viable response to the crisis.
"We needed change. Our situation has been pathetic," said Keresenzia Moyo, 65.
"The economy has been in the doldrums for a very long time. We are happy with what has been done."
However a spokesman for the ruling ZANU-PF party, Simon Khaya Moyo, insisted it was business as usual.
"It's normal, everything is normal with the party," he told AFP.
Harare's residents largely ignored the few soldiers still on the streets today and continued commuting, socialising and working.
The international community has been watching the crisis closely.
"(Problems) need to be resolved politically by the ZANU- PF party and not with an intervention by the army," added Conde.
Britain, Zimbabwe's former colonial ruler, called for elections scheduled for 2018 to go ahead.
The state-owned Herald newspaper walked a fine line in its editorial pages today by remaining loyal to Mugabe but also endorsing the military's action.
"The military does not readily interfere... they had to break with this long tradition," it wrote, adding that ZANU-PF "was being soiled by those who should be helping the President".