Seamer Dhammika Prasad removed both openers by the third over and left-arm spinner Rangana Herath also chipped in with a wicket as the tourists plodded to 38 for three by tea.
Prasad had Mohammad Hafeez caught at third slip off his sixth delivery and then trapped Ahmed Shehzad leg-before in his next over to reduce Pakistan to 11 for two.
It became 35-3 when Herath, brought in to the attack in the sixth over, had Azhar Ali leg-before in his third over.
Sri Lanka's innings was built around a painstaking century from opener Kaushal Silva, who defended solidly at one end even as Pakistan ran through the batting at the other end.
Also Read
The diminutive right-hander lit up the rain-hit Test with a stubborn 125, his second Test century after making 139 against Bangladesh in Dhaka in January last year
Silva, however, failed to carry his bat through the innings when he was ninth out from the 300th delivery he faced, caught behind off left-arm spinner Zulfiqar Babar.