Swamy, who was on a personal visit to attend an event of a university in the state last week, had reportedly said that mosques and churches are mere "buildings" to offer prayers.
"So there is no problem in demolishing them. They demolish mosques even in Saudi Arabia," he had said.
He further said that god lives only in temples.
"His comments are completely insulting for a community. We will take action. If he continues to talk like this, we will ban his entry into Assam," Chief Minister Tarun Gogoi said.
Also Read
He wondered why BJP was not taking any action against Swamy in spite of "controversial comments" made my him.
Meanwhile, Assam Police has registered an FIR against Swamy at Latasil police station here under sections sections 120(B) (criminal conspiracy), 298 (Uttering, words, etc., with deliberate intent to wound the religious feelings of any person), 153(A) (Promoting enmity between different groups on grounds of religion, race, place of birth, residence, language) of IPC on a complaint by Chatra Mukti Sangram Samiti, students wing of Krishak Mukti Sangram Samiti (KMSS).
The BJP sought to distance itself from the comments saying they were made in his "personal capacity".
"We will have a talk with him on the matter. These comments, which I have not seen, must be made by him in his personal capacity. This is not the party's stand," Union minister Prakash Javadekar said in Mumbai.
All Assam Students' Union, All Assam Minority Students Union and All Bodoland Minority Students' Union along with other student bodies burnt effigies of Swamy and Assam BJP president Siddhartha Bhattacharya in Guwahati and Kokrajhar.
Swamy had further said that there was no need to get emotional over giving a portion of land to Bangladesh under the Land Boundary Agreement.
"If the Assamese people are so sentimental about the issue, then they should sit back at their homes and shed tears," he had reportedly said.