The reform-minded new Secretary of State comes in as Francis mulls an overhaul of the way the Roman Catholic Church is run that could put an end to what the pope calls a "Vatican-centric" system.
The influential post is frequently referred to as equivalent to that of a Vatican prime minister.
Its incumbent can represent the pontiff on some occasions, and three have gone on to be popes.
Bertone was appointed by the pope's predecessor, Benedict XVI, in 2006 and proved a divisive figure who was disliked by many for his management style.
Also Read
A series of leaks of confidential Vatican papers last year by Benedict's personal butler revealed infighting between pro- and anti-Bertone factions.
Bertone's image also suffered because his time in office coincided with a traumatic period for the Vatican, shaken by revelations of widespread child abuse and scandals involving its finances.
Bertone himself lashed out at his critics in September saying he had been the victim of "moles and vipers" in the Vatican system.
"Of course there were a lot of problems, particularly in the last two years, and some accusations were levied against me," Bertone said.
"We missed some things, also because problems were kept locked away by some people who did not contact the Secretariat of State," he said.
The formal handover of power came in a closed-door Vatican ceremony today attended by top officials from the Secretariat of State.
At 58, Parolin is considered relatively youthful for top Vatican office and is the youngest cleric to occupy the post since cardinal Eugenio Pacelli, who went on to become Pius XII in 1939.