Coast guard and border guard officials said the boat was overloaded with about 100 people when it sank late Sunday in the mouth of the Naf river that separates Myanmar from Bangladesh.
Border Guard Bangladesh (BGB) official Abdul Jalil told AFP 12 bodies had been recovered after an all-night rescue operation, saying "they include 10 children, an elderly woman and a man".
More From This Section
Area coast guard commander Alauddin Nayan said the boat capsized near the coastal village of Galachar with nearly 100 people on board.
He said some 40 people in the boat were adult Rohingya Muslims fleeing their villages in Rakhine.
"The rest were children," he said.
Border guard boats have rescued 13 Rohingya including three women and two children after scouring the estuary of the Naf river, Jalil said.
Since the boat capsized near the Myanmar side of the border, Jalil said many may have swum to the Rakhine coast.
The coast guard said the boat sank at around 10:00 pm.
Local media quoted a survivor as saying the vessel capsized due to high waves and bad weather.
Nearly 520,000 Rohingya Muslims have fled Rakhine state for Bangladesh since late August, many walking for days through thick jungle before making the perilous boat journey across the Naf river.
Around 150 Rohingya, many of them children, have drowned while trying to reach Bangladesh in small, rickety fishing boats that coastguards say are woefully inadequate for the rough seas.
Last week more than 60 Rohingya refugees are feared to have died after a boat carrying them from Myanmar capsized in rough weather in the Bay of Bengal just off the Bangladesh coast.
The bodies of 23 people were retrieved, but the death toll was expected to surge, with many of the dead likely to be young children too weak to swim through the churning water.
The refugee crisis erupted after Rohingya militant raids on Myanmar police posts on August 25 prompted a brutal military backlash.
The United Nations has said the army campaign could amount to "ethnic cleansing" while Myanmar military leaders have blamed the unrest on the Rohingya.
The government of Buddhist-majority Myanmar refuses to recognise the Rohingya as a distinct ethnic group and considers them illegal migrants from Bangladesh.
While the worst of the violence appears to have abated, insecurity, food shortages and tensions with Buddhist neighbours are still driving thousands of Rohingya to make the arduous trek to Bangladesh.
Bangladesh has made the journey even more difficult with a clampdown on boats running refugees across the Naf river.