Dr Rajiv Mishra was suspended after reports that 30 children died in the hospital in 48 hours.
"Owing to laxity, the principal of BRD Medical College has been suspended with immediate effect," Singh told reporters.
The cases led to outrage in the country.
UP Chief Minister Yogi Adityanath said in a tweet, "Prime Minister Narendra Modi sought information about the Gorakhpur incident. On his directives, Union Health minister JP Nadda and MoS Anupriya Patel also sought information of the incident.