Around 10 to 12 huts were gutted in the fire which broke out at Chokha Bazaar slums in the city's Kalupur area, Police Inspector V N Yadav said.
The exact cause of the blaze was yet to be ascertained, he said.
However, it appears that someone apparently burnt rags near the slum area and due to the wind the fire spread to the nearby huts, he said.
The deceased were identified as Dinesh Chunara (15), his brother Manoj Chunara (10), relative Geeta Chunara (30) and Dhaval Kalubhai Popatbhai (12), Yadav said.
A probe in the case is on, he added.