Togadia, who hails from Gujarat and belongs to the powerful Patel community, in a statement expressed dismay over the "misuse of photographs obtained from internet" in which he is shown flanked by Hardik Patel and some other youngsters.
"I get to meet any number of people every day. Many of them like to get their photographs clicked with me to which I agree out of courtesy. But to use such photographs to create mischief is deplorable, be it in mainstream media or social media," he said.
"Ever since Gujarat has been rocked by violence, I have only appealed for peace and never targeted any political party. Such news items are being planted by those who wish to defame me.
"I have sought legal opinion and my lawyer is sending notices to deter people hostile towards me from spreading such falsehood in mainstream media as well as social media," Togadia added.