A group of 20 soldiers entered the Israeli side of the strategic plateau through the Quneitra Crossing, the only direct passage between Israel and Syria, an AFP photographer said.
About 50 other soldiers of the 378-strong Austrian contingent were to be withdrawn during the day, sources on the ground said.
Austria, which has been a cornerstone of the UN force monitoring a ceasefire between Syria and Israel since 1974, announced on Thursday that it would withdraw its peacekeepers because of deteriorating security on the Golan Heights.
A top Israeli government official told AFP yesterday that several dozen Austrian troops had already left the mission's headquarters. Israeli public radio said they were administrative staff.
"But the majority of soldiers will remain in place until the UN has found a country that can send troops to replace the Austrian ones," said the Israeli official, who asked not to be named.
The UN is trying to persuade Austria to slow down its withdrawal.