The issue came to light after a picture, in which Lord Swaminarayan can be seen donning the Sangh outfit -- white shirt tucked into a baggy khaki shorts with black cap and black shoes on -- went viral on social media platforms.
The idol is also seen holding the national flag in one hand.
Seeking to play down the controversy, Swami Vishwaprakashji of the temple, which is situated in Laskana area of Surat, said the dress was gifted by a local devotee a couple of days back and it was the normal practice of showing the idol in different dresses.
Notwithstanding denials by the temple authorities of having any intention of endorsing the views of the right-wing organisation, Congress leader Shankersinh Vaghela said they must refrain from such activities.
"What do you want to prove by dressing up the God in khaki shorts? I pity those who have done that. Today, you have dressed the Lord in RSS uniform, tomorrow you will dress the idol in BJP's uniform. This is very unfortunate," Vaghela said.