Police said the District Crime Branch arrested them following complaints from village administrative officers of Keezhaiyur, Keezhavalavu, Ayyapatti and Usilampatti.
The arrests have come even as IAS officer U Sagayam, appointed by Madras High Court to probe alleged illegal granite quarrying in the district, is probing the matter.
Sagayam, appointed by the court in September last year, had visited various sites and received petitions from public in the district since he began his work on December 3.
Police have registered 135 cases against various persons for illegal quarrying and causing revenue loss to the state exchequer for more than Rs 14,000 crore.