The officials say gunfire was heard this afternoon inside the Special Forces headquarters, located in the heart of the capital near the presidential palace.
The mutineers chanted "leave leave" and tried to storm the office of Mohammed Mansour al-Ghadraa, their new commander. Al-Ghadraa was appointed in September.
Shiite rebels who seized the capital in September deployed their militiamen, officials said, adding that Presidential Guards cordoned off the area with armored vehicles and soldiers.
It's not immediately clear whom the mutineers want to replace al-Ghadraa.
More From This Section
The officials spoke on condition of anonymity because they were not authorized to speak to the press.
Yahia Saleh's removal was part of a shakeup by Saleh's successor Abed Rabbo Mansour Hadi in response to widespread demands to purge Saleh's loyalists from the security forces and army.
Saleh and Shiite rebels known as Houthis have joined ranks in weakening Hadi, sweeping through cities and towns, and prompting the United Nations Security Council to impose sanctions on Saleh and two Houthi commanders.
Saleh was forced to step down after a yearlong uprising in 2011.
He remains a major power broker thanks to a US-backed, Gulf-brokered deal which gave his party a share of power and granted him immunity from prosecution in return for stepping down.