The victim, surnamed Liu, was with her mother and headed to a metro station to meet up with relatives when she was attacked, according to a police statement.
The suspect, a 33-year-old man surnamed Wang, reportedly overpowered the child's mother and decapitated the child with a kitchen knife.
"The girl's mother rushed forward to pull Wang away, but he was stronger and pushed (her) aside," read the statement. "She died on the spot."
Wang was arrested at the scene of the crime and is currently in police custody. He has yet to enter a plea.
Police said Wang had been booked in the past over drug-related crimes. Taiwan's Apple Daily reported that Wang was unemployed and living with his parents and had been hospitalised due to mental health issues.
Taiwan's outgoing President Ma Ying-jeou called on authorities late today to conduct a thorough investigation into the murder, while police island-wide were instructed to increase patrols of neighbourhoods.