EU President Donald Tusk said the agreement was reached after an overnight summit to show "solidarity with frontline countries."
French President Francois Hollande said the EU will also resettle 20,000 refugees, mostly from Syria and Iraq, who are currently in camps outside the EU.
The migrant crisis has caused tensions among EU countries, and Italian Prime Minister Matteo Renzi made an emotional plea to the leaders for help.
Some 2,600 have died or gone missing during the often-perilous sea crossing.
Also Read
The EU wants to oblige member countries to share the 40,000 Syrians and Eritreans requiring international protection who are expected to arrive in Italy and Greece over the next two years. But many of the 28 nations are refusing to have migrant numbers dictated to them from Brussels.
The leaders did not spell out any quotas for how many refugees each country will take. EU interior ministers will finalize the plan next month.