Gilles de Kerchove, in Paris this week for a meeting of EU counter-terrorism magistrates, said jihadists who had clearly engaged with the most brutal militant groups and risked carrying out violence within Europe must be prosecuted.
But he said authorities must also identify less fanatical members who could be open to deradicalisation.
"There is a considerable number of Europeans among the foreign combattants who have travelled to Iraq and Syria, certainly more than 4,000," said Kerchove.
"Before Syria, we were talking about dozens of people to monitor. Now we're in the thousands."
Also Read
He said there were many cases in which there was little or no evidence against jihadists returning from Syria.
"We are not present in Syria. We don't collaborate with the Assad regime, so it's not easy to prove that someone joined the Islamic State group or (Al-Qaeda-affiliated) Jabat al-Nusra rather than the Free Syrian Army," said Kerchove.
"I've said it for a long time: it would be a profound error to send everyone that returns from Syria or Iraq to prison, not least because prisons are a major vehicle for radicalisation."
"It's difficult, especially since many have learned the art of subterfuge. It's not an exact science but... We must reduce the number of suspects by putting in place systems of rehabilitation or psychological and social assistance programmes for those who do not present an objective risk."
French Prime Minister Manuel Valls announced Wednesday that authorities would by the end of the year start taking in volunteers returning from conflict zones who were not the subject of legal action.
"By an individualised approach, and psychological support... These young people should find their place in our society again," he said at the magistrates' gathering.
"I've always said that our societies need to learn to live with the terrorist risk," he said.