The offensive began on Tuesday along three fronts in the remote Mafhed mountains of Abyan and Shabwa provinces, officials said.
They say that among those killed is a foreign national from Uzbekistan named Abu Mussalam al-Uzbeki, in addition to an alleged al-Qaeda leader in the region.
The military's claims could not be immediately verified. Yemen's government previously has claimed to have killed leading militants who later turned out to be alive.
Al-Majaala witnessed one of the deadliest strikes in the ongoing US drone campaign in Yemen. In 2009, a strike there killed dozens of people and sparking an international outcry against the use of drones over civilian casualties.
More From This Section
Military officials say forces also faced stiff resistance in Shabwa as militants fired mortars and artillery at soldiers.
Washington considers Yemen's al-Qaeda in the Arabian Peninsula as the world's most dangerous branch of the terror network. The group is blamed for a number of unsuccessful bomb plots aimed at Americans, including an attempt to bring down a US-bound airliner with explosive hidden in the bomber's underwear and a second plot to send mail bombs hidden in the toner cartridges on planes headed to the US.
The US has been hitting suspected al-Qaeda positions in the country with drone strikes over the past two years, trying to cripple the group after it was driven out of several southern cities it took over in 2011.