Abubaker Shariff Ahmed, better known as Makaburi and a vocal supporter of Osama bin Laden and Al-Qaeda, was gunned down in the strategic city on Tuesday. Prior to his death he had accused Kenyan security forces of seeking to kill him.
He was also the third hardline Kenyan Islamist to be killed in Mombasa in as many years.
"Gunning down clerics in the streets is only making a bad situation worse," said Leslie Lefkow, deputy Africa director at Human Rights Watch.
The rights group said the killing of Makaburi and other Islamists "appear linked to accusations by authorities that they were involved in terrorist activities."
Also Read
The cleric, who was in his 50s, had also openly praised the suicide commandos who stormed Nairobi's Westgate mall in September last year, massacring at least 67 people in a four-day siege that was carried out in retaliation for Kenya's intervention in Somalia to fight the Shebab.
"The police have failed to investigate these killings," Human Rights Watch said, adding that Kenyan police have regularly responded to militant activity with abuses against ethnic Somalis.