The incident occurred when the labourers were filling explosives in firecrackers at the factory located at Rau tehsil, an official said.
"The blaze triggered a series of explosions," district collector P Narhari said, adding that exact cause of fire is being investigated.
He said the factory's license had expired on March 31 but it was being run illegally thereafter.
Director of the factory has been booked and further investigations are on, Narhari said.