The Italian news agency Ansa quoted the local mayor of Lampedusa, Giusi Nicolini, as saying 82 bodies had been recovered by emergency workers, including those of two children and several women.
"The survivors are in a state of shock. They have been in the water since the early hours of the morning," Nicolini told news channel SkyTG24, as emergency workers raced to rescue more survivors.
Italian Prime Minister Enrico Letta called the incident "an immense tragedy" in a tweet.
Officials said the bodies were being taken to an airport hangar because of the large numbers.
More From This Section
"We cannot keep on counting the dead. We have to mobilise, do something concrete to avoid these constant tragedies of despair," the local archbishop, Francesco Montenegro, said.
The asylum-seekers said they were from Eritrea and Somalia and local police were quoted as saying they believed the boat had left from Libyan shores.
Prosecutors have opened an inquiry for multiple murder, as well as favouring illegal immigration.
The fire then spread, sowing panic on board which caused the boat to flip over, she said.
Lampedusa is an Italian island lying between Tunisia and Sicily and is a major entryway for asylum-seekers into the European Union, with thousands arriving every year.
"The first assistance was provided by people on pleasure boats who heard the screams," Antonino Candela, a local emergency medical worker said.
The accident was the latest in many drownings involving migrants travelling on rickety fishing boats or rubber dinghies in the Mediterranean, where thousands have died in recent years.