Khalid Batarfi, a senior al-Qaeda figure who had been held for more than four years, was among more than 300 prisoners who escaped from the jail in Hadramawt province, the official told AFP.
Two prison guards and five inmates were killed in clashes, the official said.
Batarfi is among Al-Qaeda in the Arabian Peninsula's (AQAP) top regional commanders, known for his leading role in a 2011-2012 battle with Yemeni government troops during which extremists seized swathes of territory in the south and east.
Yemen has descended further into chaos since a Saudi-led coalition launched air strikes a week ago against positions held by Shiite rebels and their allies across the deeply tribal country.
Observers have warned that Yemen-based AQAP, classified by the United States as the network's deadliest franchise, could exploit the unrest to strengthen its presence in the country.