Victoria state police condemned the behaviour of both anti-racism and anti-immigration demonstrators involved in the melees in the Melbourne suburb of Coburg.
Hundreds of police attempted to keep the groups apart, and were quick to quell the clashes, but five people were arrested, with a further two held on weapons offences.
"There was riotous behaviour and it was appalling," Commander Sharon Cowden told reporters adding there were only minor injuries.
"I understand the need and the right to protest... But what we can't have is this violence in our community.
Also Read
Footage showed protesters using Australian flags on poles to beat other demonstrators, and police moving in to stop the violence.
Demonstrators from the anti-racism group hurled abuse such as "Nazi scum, off our streets" at the opposing rally at which one sign read "No refugees, our home, our future", the Australian Broadcasting Corporation reported.
Asked about the opposing groups, Prime Minister Malcolm Turnbull stressed that the multicultural nature of Australian society was "one of our great achievements".
"We are united, we have so much in common, and we are stronger and more prosperous as a nation because of that diversity," he told reporters.