England, set an unlikely target of 382 on a deteriorating pitch, were 52 for three at the close after South Africa declared their second innings on 248 for five.
Rabada, who took seven for 112 in the first innings, dismissed Alex Hales and Nick Compton, while new ball partner Morne Morkel claimed the wicket of England captain Alastair Cook for the second time in the match.
If South Africa are able to take the remaining seven wickets on Tuesday it would be a consolation victory, with England holding a winning 2-0 lead in the series.
It finished a disappointing series for Cook's latest opening partner. Hales made 136 runs in eight innings at an average of 17.00.
More From This Section
Cook did not last much longer before pushing back a return catch to Morne Morkel after scoring five.
Compton was next to go, for six, edging a drive against Rabada to be caught behind by Quinton de Kock. Compton sought a review which showed a clear edge.
It could have been worse for England. With South Africa using two spin bowlers in fading light, Root, on ten, survived a stumping chance off Dane Piedt, with De Kock fumbling a ball which turned sharply.
Hashim Amla and Temba Bavuma batted South Africa into a virtually unbeatable position with a fifth wicket stand of 117.
Amla followed up his first innings of 109 with a resolute 96, while Bavuma struck an unbeaten 78.
Amla was on 96 at tea and his approach in the first over after the interval suggested he was keen to get to his century quickly with a declaration imminent.
Stuart Broad set seven fielders on the off side. Amla tried to play across the line to leg, then flashed at a wide ball. He chased another wide delivery and was caught behind after a 199-ball innings during which he hit 11 fours.
Anderson was England's best bowler, taking three for 47. The veteran fast bowler, who had taken only four wickets at a cost of 63.50 runs each in six previous innings, struck twice in three balls early in the day, reducing South Africa to 49 for three.
Anderson had Stephen Cook caught behind for 25, then trapped South African captain AB de Villiers leg before wicket with a superb in-swinger. It was De Villiers' second duck of the match and his third in successive innings.