The blaze in the roof of the building in Bautzen, in the eastern state of Saxony, broke out overnight.
Police said no one was injured but a group of people gathered outside, some of them "commenting with derogatory remarks or unashamed joy" at the fire.
While most Germans have been welcoming toward refugees, a vocal minority has staged protests in front of refugee homes, especially in the east.
Police ordered three people to leave the fire scene because they were hampering firefighters' work and temporarily detained two of them, whom they described as intoxicated 20-year-old locals, after they ignored the order.
More From This Section
Investigators found traces of a fire accelerant at the scene and believe the fire was caused by arson, police said.
It wasn't immediately clear whether the building can be restored.
Saxony is home to the anti-Islam and anti-immigration group PEGIDA, and incidents there have caused concern before.
In August, a mob in Heidenau, outside Dresden, hurled bottles and fireworks at police protecting a shelter being set up for refugees.
Police drew criticism in that case for roughly hauling some migrants off the bus into the building - which they insist was necessary to prevent the situation from escalating - and for saying that some of the migrants had made provocative gestures.
Saxony Governor Stanislaw Tillich called the two incidents "appalling and shocking" and described the perpetrators as "criminals."
"This is abhorrent and disgusting," Tillich told the Funke newspaper group.
Interior Minister Thomas de Maiziere said that everyone in Germany is entitled to express their concerns "but there is a threshold of decency and law that must not be crossed - and this threshold was clearly crossed in the incidents in Saxony," the news agency dpa reported.