Pakistan, set a series-levelling target of 271 runs, slumped to 50-5 before plodding to 127-7 by stumps on the fourth day at the Sinhalese Sports Club.
Left-arm spinner Herath, who claimed nine wickets in the first innings, grabbed four for 46 to once again destroy Pakistan's fragile batting in front of some 5,000 screaming home fans.
Herath, who came on to bowl the ninth over after seamer Dhammika Prasad had dismissed openers Khurram Manzoor and Ahmed Shehzad, struck his first blow with his eighth delivery.
Herath then won a leg-before decision against Younis Khan to make it 50-5 in the 19th over, before Asad Shafiq and Sarfraz Ahmed stemmed the rot by batting an hour for their 55-run stand.
More From This Section
Shafiq was stumped off Herath for 32 and Abdur Rehman was leg-before to Dilruwan Perera, but Sarfraz remained unbeaten on a defiant 38.
Sri Lanka will return on Monday to take the remaining three wickets that will enable them to sweep the series after winning the first Test in Galle by seven wickets.
The hosts were bowled out for 282 in their second knock just before tea, with seamer Wahab Riaz and off-spinner Saeed Ajmal claiming three wickets each.
Sri Lankan skipper Angelo Mathews remained unbeaten on 43 after Jayawardene was dismissed for 54 in his last Test innings.
The overnight pair of Jayawardene and his long-time team-mate Kumar Sangakkara, batting together for the last time in a Test match, took their partnership to 107 after Sri Lanka resumed the day on 177-2.