Acting Chief Justice Thottathil B Radhakrishnan and Justice Anu Sivaraman upheld the decision while disposing of a batch of petitions filed by panchayats and quarry owners.
"We do not see any illegality in the single bench order," the division bench said, citing the provisions in the Panchayat Raj Act.
It also directed the panchayats to comply with the order within three weeks.
Earlier, a single judge of the high court had held that panchayats cannot insist on environmental clearance while considering applications for quarrying licence.