The police said that the truck laden with stone chips were coming from Burdwan and hit the mud house, killing the four and injuring three others.
Two of the dead children were identified as Debraj Hazra (7) and Tuku Khetrpal (8).
The vehicle lost control after hitting a sadhu riding bicycle at Sridharpur, sources said.
The sadhu was also injured and taken to a local hospital.