The left-wing Social Democrat party (PSD) filed the motion against premier Sorin Grindeanu barely six months after winning an election.
The PSD unexpectedly withdrew its support for Grindeanu on June 14, accusing him of "delays" in implementing reforms in the European Union's second-poorest country.
Grindeanu however refused to resign and denounced powerful PSD boss Liviu Dragnea for seeking to "concentrate all the power in his hands".
The 54-year-old, who led his party to a thumping poll victory in December, is barred from running for office because of a voter fraud conviction.
More From This Section
At first Grindeanu complied but recently began asserting his independence, which reportedly led to the current crisis.
Today's no-confidence motion passed with 241 to 10 votes.
The PSD and its small ally, the ALDE party, are now expected to propose a new premier to centre-right President Klaus Iohannis.
Once nominated, the incoming prime minister will then have 10 days to secure a vote of confidence in parliament for his cabinet and policy plans.
Ahead of the vote, Iohannis had urged the government to quickly resolve the crisis.
But the country of 20 million inhabitants can ill afford political instability, with the International Monetary Fund and the European Commission warning more reforms are sorely needed.
It is the second major crisis to hit the PSD since it rode back into power in December, barely a year after being forced from office over a deadly nightclub blaze.
In February, Romania's largest protests since the fall of communism forced the government to drop a bill aimed at watering down anti-corruption laws.
"Liviu Dragnea only wants one thing -- amendments to the anti-corruption laws" that currently prevent him from becoming premier, former PSD member Alin Teodorescu recently told AFP.
The PSD was forced out in the wake of angry protests in 2015 over a nightclub fire in Bucharest that killed 64 people and was widely attributed to corrupt officials turning a blind eye to fire regulations.