The accident occurred after the driver of the stone chips laden truck lost control after hitting a sadhu riding bicycle and dashed against the mud house, killing the five on the spot.
Two of the children killed were seven years old and the third one was only eight months old, the sources said.
Three persons of the same family and the sadhu were injured.
The three were admitted to a hospital where the sadhu was released after first-aid.