The duo from Melur refugee camp in Madurai district were found moving around in suspicious circumstances near the shore and gave evasive replies to questions, following which they were arrested, police said.
The ferry operator was nabbed just after he had arrived at Arichalmunai area to take the refugees back to Sri Lanka, they said.
Customs officials suspect the ferry operator might also have come here to pick up six kg of ganja from two motorcycle-borne persons, who fled on seeing the police.