Ukraine's top rebel leader announced that an offensive had begun on the strategically important port.
The report came a day after the rebels rejected a peace deal and said they were going on a multi-prong offensive against the government in Kiev to vastly increase their territory.
The move is upending European attempt\s to mediate an end to the fighting.
Mariupol, which lies on the Azov Sea, is the major city between mainland Russia and the Russia-annexed Crimean Peninsula.
Also Read
The RIA Novosti news agency cited eastern Ukrainian rebel leader Alexander Zakharchenko as saying an offensive has begun on Mariupol.
He spoke today as he laid a wreath at the site where at least eight civilians died when a bus stop was shelled in Donetsk, the largest rebel-held city in eastern Ukraine.
Rebel forces have positions within 10 kilometers (six miles) from Mariupol's eastern outskirts.
The Defense Ministry said in a statement there were three separate strikes from Grad multiple rocket launchers on Mariupol and surrounding areas.
The Donetsk region prosecutor's office said in a mid-afternoon statement that 21 people had been killed. Defense and police officials said rockets struck homes, a kindergarten, a market and shops.
There was no immediate report of how many people died in various locations.
A Ukrainian military checkpoint on a road leading out of the city toward rebel-held areas was also hit and one serviceman was killed, the Defense Ministry said.