"No human being is illegal," the crowds chanted as they walked under scorching heat past clothes shops in central Madrid, holding banners that read "Bridges not walls" and "Enough with excuses, no more barriers".
The protest, organised by dozens of NGOs including Amnesty International, comes just a few days before World Refugee Day on Tuesday.
It also comes as the deadline nears to relocate 160,000 refugees across Europe under a solidarity plan put in place in 2015 to try to remedy the continent's biggest-ever migration crisis and ease the burden on frontline states Italy and Greece.
Spain, which had pledged to take in more than 17,300, has so far only welcomed just over 1,300, according to Amnesty.
More From This Section
"Spain is doing absolutely nothing of what it should be doing, it's a disgrace -- Spain and Europe," said Carlos Diez, a 55-year-old secondary school teacher at the protest.
Also at the march was Christian Lele, a 27-year-old from Cameroon who entered Spain illegally in 2014 by climbing over a border fence between Morocco and the Spanish territory of Melilla -- one of only two land borders between Africa and Europe.
"You can't live well (in Cameroon) with the little they pay you."
Lele got to Morocco a year after he left his country, by hitching rides across the continent.
Once in Melilla, he was eventually taken to a migrant reception centre in Malaga in southern Spain, and was transferred to several others across the country before he got to Madrid.
He trained as a gardener, which is now his job in the Spanish capital.
He said he was on the march in "solidarity" with others who would follow in his footsteps.