The explosion happened at about 2:00 pm (1300 GMT) at the Jambutu Juma'at mosque in the Jimeta area of the city, shortly after the imam had finished his inaugural sermon.
Friday's blast came after at least 28 people were killed in a suicide bombing at a mosque in Maiduguri, the capital of Borno state, shortly after 5:00 am today.
It was not immediately clear whether the Yola attack was also a suicide bombing or caused by an explosive device hidden in the building.
"While worshippers had risen for the prayers to start after the sermon by the imam, there was a huge blast in the premises.
More From This Section
"There was confusion and dozens of worshippers lay on the ground in blood. We are still sifting through the victims to separate the dead from the injured.
"We're talking of scores of people dead and injured."
Another rescuer, who also requested anonymity, said: "It's bad. The casualties are definitely going to be huge.
Yola has been seen as a relatively safe haven from the Boko Haram insurgency, which has ravaged the northeast for the last six years.
Tens of thousands of people fled to the city last year, as the heavily armed Islamist group overran towns and villages across the region.
But fears were heightened after an explosive device went off at a camp for displaced people to the south of the city last month, killing seven people and injuring 20 more.