More than 200 police commandos took part in the pre-dawn raids in northern Germany to detain the men, who were suspected of either plotting an attack or awaiting orders to commit one.
The men were identified only as Mahir al-H., 17, Ibrahim M., 18, and Mohamed A., 26, in a statement issued by federal prosecutors.
They left Syria last October and travelled via Turkey and Greece -- a route used by tens of thousands of refugees and migrants -- and arrived in Germany in mid-November.
"According to what we know so far, the investigation of the (federal criminal office) BKA points to links to the attackers of Paris from November 2015," de Maiziere told a press conference.
More From This Section
"There is every reason to believe that the same trafficking group used by the Paris attackers also brought the three men who were arrested to Germany," he said, adding that their forged travel documents came from "the same workshop".
Prosecutors said in their statement that Mahir al-H. Had joined IS in its de facto capital of Raqa, Syria by September 2015 and received some weapons and explosives training.
The following month, all three men had pledged to travel to Europe in talks with an IS fighter who was "in charge of missions and attacks" outside of the Syria-Iraq region where the group has its self-proclaimed caliphate.
In Europe, "the three accused were meant to either execute a mission or await further instructions," the prosecution service said in the statement, adding that no evidence of "concrete orders or instructions" had been found.