The Sri Lankan fishermen first threatened them with hand guns and attacked them with sharp weapons, Pamban Country Boat Association leader S Arul said.
"Their fishing nets and equipment, cell phones and walkie talkies were also taken away," he said.
The fishermen were part of a group of seven who had put out to sea early this morning from Pamban, Arul said, adding the others managed to return to shore.
The four fishermen have been hospitalised.