"Modi's comment has hurt the prestige of cow protectors. He either express regret publicly within three days or we shall seek redressal from high court or apex court by filing a defamation suit," Swami Adhokshjanand Saraswati told reporters here.
He said Modi has forgotten 'Gau' (cow), 'Ganga' and 'Geeta' which enthroned him to this position.
He said instead of making efforts to save thousands of cows stuck in a Goshala in Rajasthan, the prime minister has commented against the cow protectors.
Modi's remarks came after the brutal flogging of four Dalit men by a group of cow vigilantes in Gujarat.