BJP MP and state unit General Secretary Balkrishna Shukla had filed a complaint with the Commission about Modhvadia's speech in Vadodara on November 2 when he made the controversial comments.
The Commission sent the notice to Modhwadia after listening to the CD containing his speech.
The notice said that Modhwadia's remarks prima facie violated Model Code of Conduct which prohibits personal attacks on the leaders of rival political parties.
"The Commission hereby gives notice to you to show cause as to why action should not be taken against you, for the violation of the Model Code of Conduct," said the EC notice issued yesterday and signed by its secretary Harbans Singh.
The EC has also directed Modhwadia to submit his response by November 24 and added that, "failing which it would be assumed that you have nothing to say in the matter and appropriate action will be taken against you without any further reference to you."
Modhwadia, in a public meeting on November 2 in Vadodara, had criticised Modi for repeatedly targeting Prime Minister Manmohan Singh and also took a dig at Modi for his remarks on Sunanda Pushkar, wife of Union Minister Shashi Tharoor.
He had also compared Modi with animals in the same meeting.