Modibo Keita, who has been leading peace negotiations with rebel groups, was appointed by President Ibrahim Boubacar Keita yesterday, according to a presidential decree.
Mara leaves the job after just eight months.
A source close to him told AFP that "he wanted to carry on, but it's not him who decides".
"The president wanted to change the prime minister, so the premier (Mara) issued his resignation to the president."
Modibo Keita had been appointed as the president's representative in peace talks that began in July bringing together the government and jihadist rebels based in northern Mali.