"It's now 40 dead," Ibrahim Farinloye, southwest coordinator of the National Emergency Management Agency (NEMA), told AFP by text message.
Some 124 people were rescued alive from the rubble of the building, believed to be a guesthouse for foreign followers of TB Joshua's The Synagogue Church of All Nations (SCOAN), in the Ikotun area of Lagos.
The two-storey building, located in the sprawling compound of the vast church complex, came down on Friday afternoon. It was undergoing construction work to add extra floors.
Joshua preaches to massive crowds at his megachurch in Ikotun every week and, according to his website, SCOAN "hosts thousands of national and international visitors" each year.
More From This Section
"People travel from around the world to witness and receive from the mighty work that God is doing in the life of Prophet TB Joshua," it states.
Emergency services and the media were attacked as they tried to get to the building. Rescue operations were continuing today, Farinloye said.
"The few people that were there are being rescued," he said on Friday.
Joshua was widely quoted in Nigeria's media today as saying that a low-flying plane that had been repeatedly circling the building may have been responsible for the collapse.
Ikotun lies to the west of Lagos' international airport.