The four "from the jihad scene are under investigation over suspicions that they are planning a serious act threatening the security of the state", Berlin police said.
One of the two arrested is sought by Algerian authorities for his links to IS, police told AFP.
"Investigations show that he has been trained militarily in Syria," they said in a statement.
The second Algerian was arrested for having falsified identity documents.
Also Read
Some 450 officers took part in the operation sweeping Berlin, and the states of North Rhine-Westphalia and Lower Saxony.
Since the November Paris attacks in which IS gunmen and suicide bombers attacked Paris nightspots, killing 130 people, German authorities have issued terrorism alerts on several occasions.
In November, a Germany-Netherlands football match was called off and on New Year's Eve, several Munich subway stations were closed over such threats.