The plot involved explosives and possibly knives or guns, against possible targets in the heart of the city of four million, including the area near Federation Square, Flinders Street Station and St Paul's Cathedral.
Police said they found "the makings of an improvised explosive device" in raids. The plot also involved a "surveillance mission" of the area that was to be attacked.
Prime Minister Malcolm Turnbull said: "Overnight our police and security agencies have disrupted a very substantial terrorist plot."
"Overnight, the Victoria Police, working with the Australian Federal Police..., have arrested seven persons, five of whom are still in custody, searched five premises and are continuing to search them."
More From This Section
Two of seven people initially arrested in raids on last night and this morning in Melbourne include a 26-year-old man and a 20-year-old woman who were released without charge.
Five men, aged between 21 and 26, remained in custody over the plot which has been described as "an imminent terrorist event" inspired by Islamic State.
Ashton said police carried out five raids overnight across Melbourne's northwest after a weeks-long investigation.
He added "Certainly (there was the) potential for quite a number of people to be injured or killed in this attack."
Victorian Premier Daniel Andrews described the plot as an "act of evil".
Andrews said there would be an increased police presence at large gatherings in Victoria over the Christmas period.
Police said the suspects were "self-radicalised" but inspired by Islamic State.
Justice Minister Michael Keenan said the operation marked a "successful disruption of what would have been a horrendous attack, an attack that may have caused very significant casualties."
He said the alleged terror plotters "had moved very quickly from an intention to a capability, and developed capability, including quite progressed plans."
(Reopens FGN 3)
Australia's national terrorism threat level was raised to probable in September 2014.
Since then, there have been four attacks, including the murder of a Sydney police employee last year, and 12 terror attacks have been prevented, according to Australian officials.