The accident occurred late last night when the driver of the tanker tried to avoid hitting a nilgai which was crossing the road, they said.
All the four occupants, including the driver and the cleaner, got entangled in the coal tar and died on the spot, police said.
They have been identified as Maan Singh (28), Dinesh (33), Lekhraj (30) and Manoj Kumar (28).
The bodies have been sent for post mortem, police added.