Had held the key port since 2012, in one of the biggest setbacks for the multi-national force.
The loss of Merka, the state capital of Lower Shabelle, is one of the most dramatic reverses for the AU force in its nearly decade-long battle against the Shebab.
"The AU troops pulled out of the town and Shebab militants entered - and have secured control without fighting," Ibrahim Adam, governor of the Lower Shabelle region told reporters.
The historic port, some 100 kilometres south of the capital Mogadishu, was captured in August 2012 by African Union troops.
More From This Section
There was no immediate response from the AU force, known by its acronym AMISOM, or government officials.
Residents confirmed the takeover, saying heavily armed Shebab fighters swept into the town with Islamist black flags, before addressing residents.
"AMISOM forces moved out at midday and the local administration and all other Somali security forces left a few minutes later - and then heavily armed Shebab militants entered the town," said Ibrahim Mumin, a local.
"They have been addressing residents at the district headquarters."
One passenger, named by the government as Abdulahi Abdisalam Borle, was killed, apparently sucked out through the ragged hole ripped in the metal.
The Shebab have not claimed responsibility so far and Minister for Transport Ali Jama Jangeli has called on people to await the results of the investigation.
But the seizure of Merka, as well as recent high profile Shebab attacks on AU bases, appears part of an offensive after months of apparent retreat, amid reported splits between factions allied to Al-Qaeda and some splinter groups favouring Islamic State.