The brazen attack on the sprawling complex follows a spate of hit-and-run strikes on military personnel and officials, as the country struggles to complete a thorny political transition.
The attacks in the capital and in the country's south have been blamed on al-Qaeda in the Arabian Peninsula (AQAP), which Washington regards as the jihadist network's most dangerous branch.
"At least 25 people have been killed," said a security official, shortly after the defence ministry had put the death toll at 20.
The were at a hospital that lies within the ministry's complex, which bore the brunt of the attack.
More From This Section
"A car bomb driven by a suicide bomber forced its way into the western entrance of the ministry complex," a security official told AFP.
"It was followed by another car whose occupants opened fire at the complex of buildings," he said.
The attack comes as Defence Minister Mohammed Nasser is heading a military delegation on a visit to the United States.
"The assailants took advantage of some construction work that is taking place to carry out this criminal act," it said without elaborating.
A security source said another apparently coordinated attack had been launched and a gunfight was raging outside the complex early afternoon.
State television aired gory footage of torn up bodies of what it said were suicide bombers, strewn in hospital's yard amid charred vehicles.
It also showed immense destruction within the hospital as well as floors stained with blood.