Union general secretary Bheki Ntshalintshali said it was time for Zuma to "step down" after his purge last week of cabinet critics, which included the removal of the respected former Indian-origin finance minister, Pravin Gordhan.
Gordhan's sacking contributed to a credit ratings downgrade to junk status on Monday by Standard & Poor's as pressure on Zuma grew over his move to oust opponents within the cabinet.
"We no longer believe in his leadership abilities," Ntshalintshali said at a media briefing.
Cosatu, along with the South African Communist Party and the ANC, was at the forefront of the effort to dislodge white-minority rule in South Africa that led to non-racial elections in 1994.
More From This Section
It has openly backed Zuma's deputy, Cyril Ramaphosa, who led Cosatu during the anti-apartheid struggle, to succeed him in 2019 when the president must stand down.
"Even if it means marching into the street we will do that to make our point. We believe in this alliance led by the ANC but we want a reconfiguration of this alliance," said Cosatu president Sidumo Dlamini.
"We acknowledge yesterday's announcement was a setback... but now is not a time for despondency," he told a media conference.
"What these reviews highlight is that we need to reignite our nation's growth engines."
S&P said the cut to below investment grade reflects "heightened political and institutional uncertainties" following Zuma's shock purge of critical ministers.
The shake-up has "put at risk fiscal and growth outcomes", the ratings agency added as it downgraded South Africa to BB+ from BBB-.
The rand fell three percent against the US dollar following Monday night's downgrade. The rand was trading at 13.86 against the greenback at 1045 GMT on Tuesday.
Moody's ratings agency too said it was placing its own South Africa rating -- two notches above "junk" status -- on review for a downgrade, a decision it said was "prompted by the abrupt change in leadership of key government institutions".