District officials said two pilgrims died on the spot and three, including the woman in the hospital.
The stampede occured at around 6 pm when there was a huge rush of pilgrims to board vessels to reach Kolkata, the officials said adding all of them were middle aged and are yet to be identified.
A search has been launched by naval divers to find out anybody who might have fallen into Buriganga river during the stampede, they added.