There were no immediate reports of anyone attempting to enter the complex, and there was no claim of responsibility.
The Al-Qaeda-affiliated Shebab have often used cars driven by suicide bombers as a way of breaking into a location, with more attackers following on foot.
"Five civilians died in the explosion and 13 others (were) wounded," said Somali security official Ahmed Dahir.
"It was a car bomb but their target is not clear."
More From This Section
"The explosion occurred between the main entrance of (Villa Somalia) and a nearby hotel," said a witness called Ali Ahmed.
"I saw clouds of smoke up in the sky, it was very heavy."
Expelled from Mogadishu in mid-2011, the Shebab still hold large swathes of the countryside which they use as launching pads for their attacks.
They are fighting to overthrow the internationally-backed government in Mogadishu, which is protected by 22,000 African Union AMISOM troops from Burundi, Djibouti, Ethiopia, Kenya and Uganda.
The last attacks by suspected Shebab militants were on August 22.
In a second attack the same day, four civilians died when a car bomb went off near a police station in Mogadishu.