"According to local authorities there were about 50 people at the site" at the time of the accident, the government spokesman's office told AFP late today.
Rwandan Minister of Disaster Management Seraphine Mukantabana earlier told AFP: "So far, the tentative information we have indicates that debris fell on about 100 people. We are yet to come up with the exact figures."
Police spokesman Theos Badege said three workers were confirmed dead and 21 other people had been taken to hospital.
Police were working with the army to secure the site and were "currently undertaking an intensive rescue operation", a police statement said.
More From This Section
"We are are unable to confirm details as to the cause of the building's collapse, but this will be subject to an urgent investigation, undertaken with local and national authorities, that will begin as soon as the rescue operation concludes."
The building, on a roadside in the town of Nyagatare, some 100 kilometres northeast of the capital Kigali, collapsed today at around 3:00 pm (1300 GMT), police said.
"The building that collapsed was a four-storey located in Nyagatare district by the roadside, and that is why it affected many people."
Rwanda's disaster management ministry said in a statement it had "deployed equipment to assist with the rescue operation".