The 40-year-old, who lives in Berlin and is a Turkish citizen, was detained by a special police unit in the German capital Friday, a spokesman for the city public prosecutor's office said.
He is suspected of having received weapons training in Syria and taken part in armed combat, the spokesman said.
German security services say some 400 German citizens have joined jihadists in Syria and Iraq, of whom about 130 have since come home.
Germany earlier this month announced it had outlawed active support of Islamic State including the recruitment of fighters and social media propaganda.