The 15-million pound Central Gurdwara in Glasgow is due to open to worshippers in the coming months.
Its committee found the words 'Islam No Sharia!' next to a Nazi swastika symbol daubed on one of its walls in green paint and reported the matter to the police last week.
Scotland police has now appealed for information from the public.
Chief Inspector Simon Midgely, the area commander for Glasgow West End, said: "There is no room in our society for racial abuse and it is not acceptable to treat people in such a manner.
More From This Section
"If anyone has any additional information or knowledge, please do get in touch with us," he added.
Police believe the actual incident, being treated as vandalism, took place in the last week of March.
"The Sikh community completely abhors the hateful ideology of Islamophobia. We are in complete shock that such disgraceful words were put on the walls of this great Gurdwara," said Surjit Singh Chowdhary, vice-president of Central Gurdwara Singh Sabha.
"This sad incident should energise our political leaders and fellow citizens to continue the campaign to root out such hateful beliefs. We will continue our dialogue with the police, local and national politicians to create an inclusive society and celebrate the contributions made by Scottish Sikhs to our country."
The Labour leader for Scotland, Jim Murphy, also joined in the appeal on Twitter: "The Sikh community is a vital part of Scotland. It's shameful that the iconic gurdwara has been vandalised. Call the police if you have information.