Israel's emergency services said the incident took place near Damascus Gate, the main entrance to the Old City from east Jerusalem, with the victim taken to hospital in a stable condition after being stabbed by a screwdriver.
Police said the 32-year-old had been stabbed by a "young Arab, who then fled", and that a search was under way for the assailant.
After he was stabbed the victim walked into the Old City, where he found border police and informed them of the incident, a statement said.
Violence has been raging in Jerusalem since July, with clashes between Palestinians and Israeli forces on a nearly daily basis and Palestinians pelting the city's light rail with stones.
Also Read
Tensions have soared in recent weeks with Israel pushing plans to build new Jewish settlements in east Jerusalem and Jewish extremists demanding the right to pray at the flashpoint Al-Aqsa mosque compound.
Four Israelis were recently killed in two separate attacks in Jerusalem in which Palestinians deliberately rammed their cars into groups of pedestrians and police.