South Africa were 137 for four at the close, still 166 runs behind England's first innings total of 303.
Broad dismissed opener Stiaan van Zyl as well as South Africa's two key batsmen, captain Hashim Amla and AB de Villiers, to finish the day with three for 16 from ten overs.
Opening batsman Dean Elgar withstood the Broad barrage and was unbeaten on 67, although he could have been out for 58 if England had sought a review after an appeal for leg before wicket by Ben Stokes had been turned down.
Fast bowlers Morne Morkel and Dale Steyn finished with four wickets each as England lost six English wickets in an extended morning's play.
More From This Section
Morkel took all of his wickets on Sunday, including a burst of three wickets for no runs off six balls, and finished with four for 76, while Steyn took four for 70.
Morkel's victims included top scorer Nick Compton, who made a painstaking 85 off 236 balls.
Fresh from making a hard-hit 32 not out in a last wicket stand of 36 with Steven Finn, Broad struck with the second ball of the South African innings when Van Zyl failed to play a shot against a ball which cut back and hit the top of his off stump.
South Africa would have been in worse trouble if AB de Villiers, on 11, had not survived an umpires' review after edging Steven Finn low to Ben Stokes at gully.
Stokes was not certain whether the ball had carried, an opinion shared by television umpire Bruce Oxenford after viewing several replays.
De Villiers went on to make an accomplished 49 before Broad returned to the attack and had him caught behind, pushing at a ball which left him off the pitch to provide a second catch for Bairstow.