The blaze broke out yesterday night at a prison in the northern city of Barranquilla, said the official, Saul Torres.
He said four prisoners died of the burns at a hospital and the other five fatalities remained inside the prison.
Early indications are the blaze started when inmates set fire to mattresses during a fight.
Anxious relatives crowded outside the gates of the facility, built to house 400 prisoners but home to some 1,200.