Police last week said they had thwarted an alleged imminent attack after they arrested two men, both of whom arrived in Australia in recent years, during a raid in Sydney.
"It's clear to me, that for too long, we have given those who might be a threat to our country the benefit of the doubt," Abbott said in a statement.
"There's been the benefit of the doubt at our borders, the benefit of the doubt for residency, the benefit of the doubt for citizenship and the benefit of the doubt at (welfare agency) Centrelink."
Investigators on Friday said police and a prayer hall were among potential targets.
Also Read
Al-Kutobi is suspected of entering Australia with fake documents in 2009 and was granted citizenship in 2013. Kiad arrived in Australia in 2012.
It has also emerged that Iranian-born self-styled cleric Man Haron Monis, who held a deadly siege in a Sydney cafe in December, was granted a visa in 1996 despite Tehran's warnings about his criminal past.
"In the courts, there has been bail, when clearly there should have been jail," Abbott said.
"We are a free and fair nation. But that doesn't mean we should let bad people play us for mugs, and all too often they have. Well, that's going to stop."
Abbott did not detail what changes were planned, but said he would deliver a national security statement a week on Monday.
Australia in September raised its terror threat level and carried out extensive raids in Sydney and Brisbane to disrupt an alleged plot by IS supporters to abduct and behead a random member of the public.
In December Monis took 17 people hostage for some 16 hours at a cafe on central Sydney, rattling Australia.