"During a special operation in the village of Nardaran, 32 radical Islamists were arrested and five were killed," Usubov said in a televised statement last night.
The men were all members of the Movement for Muslim Unity, which allegedly sought to stage a coup and "establish a Sharia state in Azerbaijan", he added.
The group has been active for more than a year in Nardaran, located just outside the capital Baku, Usubov said.
Two policemen and four alleged members of the Movement for Muslim Unity were killed on Saturday in a shootout during a police raid in Nardaran.
Nardaran - one of Azerbaijan's poorest settlements - has a reputation for being a stronghold of conservative Shia Muslims and has been the site of regular anti-government protests.
Shia-majority Azerbaijan is regarded as one of the Muslim world's most secular countries.