About 10 lawmakers from the Economic Freedom Fighters (EFF) party shouted down the Speaker and refused to let Zuma deliver his budget speech, saying they did not recognise him as president.
Plain-clothed security guards pulled the EFF members from their seats and bundled then out of the chamber as punches were exchanged and parliamentary benches knocked over.
The country's highest court in March found Zuma had violated the constitution over the use of public funds to upgrade his private residence.
"We cannot allow an illegitimate president to speak here as if nothing has happened," EFF deputy president Floyd Shivambu told parliament during chaotic scenes.
More From This Section
"We cannot be addressed by a person who is facing corruption charges (and) undermines the constitution. He must not speak here."
Criticism of Zuma has built over recent months with several stalwarts of the anti-apartheid struggle calling for him to step down, but he retains the loyalty of many in the ruling African National Congress (ANC) party.
Zuma calmly delivered his speech to parliament after the EFF members were removed.
The president has also been beset by allegations that a wealthy Indian migrant family had such influence over him that it could decide ministerial appointments.