The children were killed when their school bus was caught up in the attack targeting the home of a leader of the Shiite militia, known as Huthis, in the town of Rada, a local security source told AFP, blaming the attack on Al-Qaeda militants.
A medical source confirmed 25 people had died in the attack.
Yemen has been rocked by instability since the Shiite fighters, known as Huthis, seized control of the capital Sanaa in September.
Today's bomb attack was the second to target Huthis in Rada in little more than a month.
On November 12, a suicide bomber killed dozens of people gathered at the residence of a tribal chief in Rada.
The mixed Sunni-Shiite town has seen heavy fighting since the Huthis took over parts of it in October.