The moves follow similar decisions earlier this week by Britain, France and the United States regarding their embassies in Sanaa, the Yemeni capital.
"The situation is anything but stable," said a German foreign ministry spokeswoman, calling the ouster of the government by Shiite militiamen known as Huthis "unacceptable, dangerous and with consequences for the region".
"We decided yesterday to temporarily close the embassy in Sanaa and the personnel departed the country early this morning," she said.
UN Secretary General Ban Ki-moon said yesterday that Yemen was "collapsing before our eyes" towards civil war.