The local currency, the rand, fell sharply to 15.24 to the dollar shortly after the announcement, extending a series of losses this year.
"I have decided to remove Mr Nhlanhla Nene as Minister of Finance, ahead of his deployment to another strategic position," Zuma said in a statement, without specifying what that position would be.
"Mr Nene has done well since his appointment as Minister of Finance during a difficult economic climate," Zuma said, without providing the reason for sacking him.
Van Rooyen was described by newspaper Business Day as "a malleable member of parliament's finance committee".
More From This Section
Nene's departure had been predicted earlier today by Business Day, which pointed out that the Treasury under Nene had "dragged its feet" in providing funds for a massive nuclear power deal which critics say could bankrupt the country.
Nene had also clashed recently with Zuma acolyte Dudu Myeni, the board chairman of the ailing national airline South African Airways, which relies on government bailouts to keep flying.
The announcement of the change at the top of the finance ministry comes as ratings agencies rank government debt barely above junk status, with unemployment at more than 25 percent and South Africa narrowly escaping recession with 0.7 percent growth in the third quarter.