Elgar was dismissed by off-spinner Dilruwan Perera in the truncated start-stop first session that saw the visitors add 17 runs off seven overs.
The downpour led to an early lunch with the Proteas placed at 55-2 while chasing a target of 369. They require 314 more runs with eight wickets in hand.
Quinton de Kock, promoted up the batting order, was batting on 28 with first-innings centurion and skipper Hashim Amla on four at the break.
Sri Lanka declared their second innings on 229-8.
More From This Section
Perera (1-27) gave the side an early breakthrough when Elgar failed to judge the ball that pitched on the middle but turned to clip the off-stump.
Elgar made 13 after facing 65 balls and shared 43 runs for the second wicket with De Kock.
Elgar's opening partner Alviro Petersen was out for a duck late Sunday, being caught by Kithruwan Vithanage off left-arm spinner Rangana Herath (1-7).