The stone arch bridge in Gaozhou City collapsed at 1 pm local time, said the city's publicity officials.
Rescuers pulled out 26 people from the debris. Of them five died and eight others were seriously injured, state-run Xinhua news agency reported.
An initial investigation showed the bridge was not licensed and was built by the village on its own.
The contractor for the construction has been taken into custody.