"The legitimate government of Yemen is recalling charge d'affaires Abdullah al-Sirri," a statement from the press office of President Abedrabbo Mansour Hadi said.
Hadi fled to Riyadh in late March as the Iran-backed Huthis advanced on his refuge in Yemen's port city of Aden.
A Saudi-led coalition then began bombing the rebels and their allies in an attempt to halt their advance, but heavy combat continued on the ground between the rebels and forces loyal to Hadi.
Tehran has dismissed as "utter lies" accusations that it armed the rebels.
Sunni-dominated Saudi Arabia feared the Huthis would occupy all of Yemen and move it into the orbit of its regional rival Iran.