"We arrived at the scene and managed to remove three bodies," said Victor Claros, head of the firefighters in the Colombia's Cauca state, where the disaster took place.
Two people pulled alive from the mine had been taken to hospital, he added, speaking to Caracol Radio.
But Claros could offer little hope for those who were not pulled to safety immediately after the cave-in, saying that some of the workers were buried under 20 metres of earth.
Claros said the accident happened at a mine outside the city of Santander de Quilichao, where independent mine workers were excavating without authorisation.
Also Read
Colombia has more than 14,000 mines, more than half of which operate without proper permits, officials said. The government even has confiscated heavy excavation equipment at some illegal sites.
Justice officials said the mines, illegally scoured for their minerals, pose a risk to the authorized workers who enter them.
"The conditions of informal labor and informal infrastructure pose a latent risk to workers," said a statement released today by the public defenders office, which is tasked with defending the rights and welfare of Colombians.