With the political debate about Islam heating up ahead of Britain's general election next month, the impact is particularly keenly felt in Birmingham -- a city where a quarter of the population is Muslim.
"The politicians are using Muslims as a punch bag," said Abdul Rashid, head of the city's main mosque.
"The politicians look for cheap and easy ways to popularity... And at the moment the cheap and easy way of gaining popularity is immigration and Islamophobia," the 73-year-old told AFP.
Birmingham is Britain's second biggest city with 1.2 million residents -- almost half of them from ethnic minorities -- and is often singled out when fears are expressed about a growing Islamist threat.
Following the Islamist attacks against the satirical weekly Charlie Hebdo in Paris in January, an analyst on the US network Fox News even spoke of Birmingham as a "no-go zone" for non-Muslims.
He since apologised for the comments, which hurt many inhabitants in the former industrial hub.
The first Muslim immigrants -- Yemenis -- arrived in Birmingham in the early 20th century to work in the metal industry. There have since been waves of arrivals, many of them from the Indian subcontinent but also from Afghanistan, Bosnia and Somalia.
The city is known now for a mostly peaceful coexistence between communities, although it was hit by a nationwide spate of riots in 2011 following the police killing of a mixed-race man in London.
Birmingham is "a city where people feel that they can live together and tolerate each other," said Mashuq Ally, deputy head of a special local government department in charge of community cohesion.
Ally is in constant contact with the city's dozens of ethnic communities to avoid any disputes or violence.
"Although we are said to be the second most vulnerable city to terrorism after London, we are fairly safe because the community is part of the solution," he said, hailing cooperation between the city council, Muslim communities and the police.
Philippe Trzebiatowski, a young Frenchman working on a Birmingham travel guide for the company Expedia, said that "people mix quite naturally here".
Not everything is rosy however in Brummie-land -- the diminutive nickname used for city residents.
