The brazen attack on the sprawling complex follows a spate of hit-and-run strikes on military personnel and officials, as the country struggles to complete a thorny political transition.
The attacks in the capital and in the country's south have been blamed on al-Qaeda in the Arabian Peninsula (AQAP), which Washington regards as the jihadist network's most dangerous branch.
Also Read
"At least 20 people have been killed in the attack," the defence ministry said in a brief statement, without releasing a toll for the wounded.
"A car bomb driven by a suicide bomber forced its way into the western entrance of the ministry complex," a security official told AFP.
"It was followed by another car whose occupants opened fire at the complex of buildings," he said.
The attack comes as Defence Minister Mohammed Nasser is heading a military delegation on a visit to the United States.
The ministry said gunmen occupied the Defence Hospital, within the complex, after the explosion, but security forces had regained control of the building.
"The assailants took advantage of some construction work that is taking place to carry out this criminal act," it said without elaborating.
In an apparently coordinated attack, a security source said a gunfight was still raging outside the complex.
Plumes of smoke billowed across the ministry complex, situated on the edge of the Baba al-Yaman neighbourhood, as gunfire was heard.
"I heard a series of explosions, and then an exchange of fire," a wounded soldier told AFP.
State television broadcast appeals for blood donations at hospitals treating the wounded.
Yemen has been going through a difficult political transition since veteran president Ali Abdullah Saleh was ousted in February 2012 after a year of deadly protests against his 33-year rule.