The death toll was rising in the historic city of Tainan, which bore the brunt of the 6.4-magnitude quake, as rescuers scoured rubble for survivors.
An entire residential complex of four buildings containing almost 100 homes toppled, left on its side with twisted metal girders exposed and clouds of dust rising from the jumbled concrete.
Among the nine people killed when the apartment complex collapsed were a 10-day old baby girl and two other children, the National Fire Agency said.
Officials say they are unsure how many residents may still be inside, but media reports estimated at least 30 could be trapped.
Also Read
Around 800 troops have been mobilised to help the rescue effort, with sniffer dogs also searching through the rubble.
Residents at the felled 16-storey Wei-kuan Building told of their terror as the quake hit, with survivors pulled bleeding and crying from the ruins, some just in their underwear.
"I saw buildings shake up and down and left and right," said one resident.
Another man tied his clothes together to create a rope and lowered himself from his home on the ninth floor to the sixth floor below, Apple Daily reported.
One woman told how she had fought her way out of her home.
"I used a hammer to break the door of my home which was twisted and locked, and managed to climb out," she told SET TV, weeping as she spoke.
Rescuers have freed more than 250 people from the apartment complex, with over 40 of them hospitalised.
"We are concerned that most members of those families may have returned for the coming new year holiday," he said.
Heartfelt appeals for the missing were posted on social media.
"My friend in Wei-kuan is currently missing. His brother is waiting at the scene and other relatives are at the hospital looking among those injured. If anyone has related news, please get in touch," one user called applexgreen posted on Taiwan's popular PTT forum.