The incident, which occurred on Saturday, came to light when the woman along with her husband filed an FIR with Gopalganj police station.
Police today said the three sweepers followed her when she went to the toilet and allegedly raped her after threatening to take her life.
The woman had gone to the district hospital for treatment of her baby.
City Superintendent of Police Pramod Sonkar said that all the three accused -- Rahul, Sunny and Sachin -- were arrested.
Condemning the incident, Leader of the Opposition in Madhya Pradesh assembly Ajay Singh today said that Chief Minister Shivraj Singh Chouhan had talked about guaranteeing the safety and dignity of woman but "he has failed to do anything in this regard" and demanded that the government take steps to ensure that such incidents do not recur.