The 66-year-old announced at a party meeting that he would stand down from all his positions of responsibility in the party after it suffered a convincing election defeat, a party source told the German DPA news agency.
"My political life will come to an orderly end," he told fellow members of the Social Democratic Party (SPD) in Berlin.
The announcement came as the party was debating whether it would consider joining a "grand coalition" with Merkel's victorious conservative Christian Democrats (CDU).
Around 200 SPD members are now meeting behind closed doors to discuss the party's next move after it came second in the polls with just under 26 percent.
Also Read
Steinbrueck, the former premier of populous North Rhine-Westphalia state, was chosen as the candidate to challenge the popular Merkel after other top players excused themselves from what many saw as a political suicide mission.
The trained economist had earned kudos as a finance minister under Merkel in a 2005-2009 "grand coalition" between the two parties.
He was also accused of being out of touch with voters after he sneered at modestly priced Pinot Grigio wine and pointed out that chancellors earn little more than regional bank managers.