Williamson was not out 78 as he propped up the innings with nightwatchman Jeetan Patel on nine, taking New Zealand to 177 for three in reply to South Africa's 308.
Taylor was on eight when he limped from the field during a torrid period in which he was also hit on the helmet by a fired-up Morne Morkel, who sent down a lively 10 overs for 26 in his comeback from a serious back injury.
It was a rapid end to an innings where the wicket offered little for the bowlers and Dean Elgar, with his 140, had the tourists well placed at 252 for four.
By tea, New Zealand were 59 for one and they added a further 118 for two wickets, plus the loss of the injured Taylor, in the final session.
Also Read
South Africa removed Tom Latham early, caught behind off Vern Philander for 10, and New Zealand were 15 for one.
The opener had offered early chances with close-run singles before he settled down and looked set by the time he brought up his half-century driving Keshav Maharaj for the sixth four of his innings.
But he lasted only another seven balls and was unable to score again before he scooped the spinner to Elgar at midwicket.
Disclaimer: No Business Standard Journalist was involved in creation of this content