The centre-left Faymann, 56, chancellor since 2008, said in a statement that he no longer had "strong backing" in his party, the Social Democrats (SPOe).
"As a result of this insufficient support I am drawing the consequences and resign my functions as party leader and chancellor, effective today," he said.
The SPOe and its coalition partner since 2008, the centre-right People's Party (OeVP), have dominated Austrian politics since World War II but their support has been sliding in recent years.
Mirroring similar trends across Europe, the two main parties have been bleeding support to fringe groups, in Austria's case in particular to the far-right Freedom Party (FPOe), the former party of the controversial, late Joerg Haider.
More From This Section
The far-right has tapped into growing unease about immigration after Austria last year saw 90,000 people claim asylum, and around 10 times that number pass through at the high point of Europe's migrant crisis.
The FPOe is leading national opinion polls and on April 24, in the first round of elections to the largely ceremonial post of president, the FPOe's Norbert Hofer came a clear first with 35 percent.
Hofer, 45, who presents himself as the friendly and reasonable face of the FPOe, will now face Alexander van der Bellen, a former head of the Greens who came second, in a runoff on May 22.
This historic failure means that for the first time since 1945, there will not be a president from within these two parties in Vienna's Hofburg palace.
This in turn could mean that the new president might make use of some of the considerable powers afforded to the head of state under Austria's constitution that until now have been not been used.
In theory the Austrian president can fire the government -- as Hofer has threatened to do if elected -- or dissolve parliament.