Wahab Nejib Ahmed Aoun and a relative were gunned down by two men on a motorbike in the central district of Mansura where the prison is located.
The incident comes a week after a similar attack claimed the life of a police colonel in the city.
This method of attack is often used by Al-Qaeda, which has operated for years in south and southeast Yemen, and is currently the target of a huge offensive by government forces.
Yemen-based Al-Qaeda in the Arabian Peninsula (AQAP) is regarded by Washington as the network's most dangerous branch.
Also Read
It had taken advantage of the government's war against Shiite rebels to spread its presence in the south.
AQAP was ejected recently from Aden neighbourhoods including Mansura and also from cities in neighbouring provinces -- Huta in Lahj and Mukalla in Hadramawt.
Yesterday, AQAP began withdrawing from Abyan provincial capital Zinjibar and the nearby city of Jaar following tribal mediation to spare them from destruction.
Yemen's internationally recognised government has declared Aden the country's temporary capital since Iran-backed Shiite rebels took Sanaa and control much of the north.