The cylinder burst near the Mount Carmel school when children had gathered to buy balloons for a cultural festival there, the sources said.
The injured were admitted to the Jawaharlal Nehru Medical College and Hospital, where the condition of two was critical, they said.
Senior BJP leader and Bhagalpur MP Syed Shahanwaz Hussain visited the hospital and inquired about the condition of the injured.