The visitors, replying to Sri Lanka's first innings total of 494, were all out for 312 on the third day, trailing the hosts by a massive 182 runs.
Herath took the final wicket just two balls into the last session before rain stopped play, finishing on 3-72. Off-break bowler Perera chipped in with 3-53.
Bangladesh skipper Mushfiqur Rahim made a dogged half-century and shared 106 runs for the seventh wicket with Mehedi Hasan, ensuring the visitors avoided the follow-on.
Perera broke Mushfiqur's stubborn partnership with Mehedi when he trapped the young all-rounder plumb leg-before for 41 runs.
Also Read
The spinner also took the wicket of Taskin Ahmed off the next ball with a successful review after his appeal for lbw was turned down by umpire Marais Erasmus.
Mushfiqur and Mehedi's efforts came after Bangladesh lost four wickets in the opening session that had threatened to derail their innings completely.
"Our boys stuck to the plan extremely well and came out well this morning with the determination to try and win that first session, which they did well," said Sri Lanka coach Graham Ford.
Fast bowler Suranga Lakmal took the key wicket of opening batsman Soumya Sarkar for 71 after Bangladesh resumed the day on 133 for two.
Bangladesh lost their first wicket when overnight batsman Sarkar top-edged Lakmal's rising delivery to Lahiru Kumara at fine leg.
Soumya added just five runs to his overnight 66, his 137-ball knock containing eight fours and a six.
"If we had given 100 per cent earlier, then we have to give 120, 150 per cent to bring the match back in our favour to win," Soumya said.
Mahmudullah (8) and wicketkeeper-batsman Liton Das (5) also threw their wickets away with loose shots.
Disclaimer: No Business Standard Journalist was involved in creation of this content