Duminy scored 30 off 23 balls and Miller hit a 21-ball 36 after the tourists, who elected to bat after winning the toss, were reduced to 48-3 in the ninth over.
South Africa need a win to wrap up the three-match series, having won the first game by 12 runs in Colombo on Friday.
The tourists were unchanged for the second game, while Sri Lanka brought in Mahela Jayawardene and Nuwan Kulasekara in place of Tillakaratne Dilshan and Jeevan Mendis.
Opener Quinton de Kock scored 19 and star batsman AB de Villiers was run out after making 15.