Two men riding a motorbike shot Colonel Abdullah al-Ribaki last evening in a residential area of Mukalla, the source said, adding: "Al-Qaeda is behind this killing."
Another local official told AFP that Ribaki had been a target for the network, which recently posted notices in Mukalla threatening to step up their fight against intelligence officers and police.
Al-Qaeda militants were driven out of most of their strongholds in Yemen's south last June in an all-out offensive by government forces aided by local "resistance committees".
AQAP fighters seized large swathes of territory across the south, including most of Abyan province, which they controlled for a year.