Opener Alex Hales made a career-best 112 and nearly half of England's runs. Ben Stokes was the next highest scorer with 29.
With the series on the line, South Africa took wickets regularly, including two in two balls by Kagiso Rabada that removed danger men Stokes and then Jos Buttler for a first-ball duck.
Rabada had 3-34, allrounder David Wiese returned 3-50 and Imran Tahir 3-53 as England struggled.
England led the series 2-0 before losing the last two games to take it to a decider in Cape Town.