Victoria Police Assistant Commissioner Ross Guenther did not name the man, who has not yet been charged, and would not confirm reports he was linked to far-right groups.
But he told a press conference that the individual had a "number of affiliations to different organisations".
Guenther said police were acting on information that a person was either advocating harm against individuals or producing documents or materials which could incite harm.
"I can say that the threat was specific enough to cause us alarm, in terms of what risk it would represent to the community," he said.
More From This Section
Guenther stressed there was no ongoing risk, but said the raids on two places in Melbourne and two properties in rural Victoria had occurred because police believed "the threat to members of our community was escalating".
"I think we've interrupted what could have been something that's quite serious in terms of harm to our community," he said.
The Australian government lifted its terror threat level to high in September 2014 and recently said nine terror events had been prevented since then.