"Divers pulled out five bodies from the river in the morning," Officer-in-charge of Golachipa police station Fakhruddin was quoted as saying by the Daily Sun.
At least 25 passengers are still missing.
"The members of Fire Service and Civil Defence and Bangladesh Coast Guard retrieved the bodies between 6:00am and 12:30pm today," said Sayemuzzaman, chief executive officer of Galachipa union in Patuakhali district.
Meanwhile, a rescue vessel left Barisal for Golachipa to salvage the sunken ferry, officials at Patuakhali River Port said.
Local residents said nearly a dozen people managed to swim ashore while the fire service, police and local residents launched a campaign with boats in search of the missing.