The activists, who numbered around 40 to 50, protested outside his residence for about 20 minutes and shouted slogans against Ramesh's remarks.
They also sought his immediate apology saying his remarks have hurt the sentiments of Hindus.
Ramesh had on Friday said that India has more temples than toilets.
