55-year-old Rudd staged the comeback by winning a caucus vote by 57 to 45, three years after Gillard toppled him in a similar showdown to become the first woman prime minister.
Gillard, a strong supporter of Indo-Australia ties who took personal initiative to reverse her party's policy on uranium sale to India, said she would stand by a pledge to quit from politics following the defeat.
"Three years ago I had the very great honour of being elected as Labor leader... This privilege was truly humbling," she said.
Former Prime Minister Rudd, however, did not address media after he exacted revenge and won back his job.
More From This Section
According to reports, Gillard has written to Governor- General Quentin Bryce to ask her to commission Rudd as new Prime Minister.
She also congratulated Rudd on his win, the reports said.
It is unclear whether Rudd will stick to Gillard's schedule of the September election or go for an earlier one. The earliest date Rudd can call an election for is August 3.
Ahead of the ballot, Rudd said that tens of thousands of "ordinary Australians" have been asking him to take back the leadership.
Today's leadership test was the third faced by Gillard since she took office in 2010. Rudd himself launched an unsuccessful challenge in early 2012, but was routed 71 votes to 31.
A major reshuffle of the cabinet is expected.
Deputy Prime Minister and Treasurer Wayne Swan resigned after Rudd's victory and was replaced by Rudd ally and Transport Minister Anthony Albanese in a second ballot.