During a heated exchange about his economic credentials the conservative Abbott said criticism coming from Labor was like "the arsonist complaining about the fire".
He went on to say opposition leader Bill Shorten was: "The Dr Goebbels of economic policy," likening him to Joseph Goebbels, minister for propaganda in Nazi Germany.
Abbott quickly withdrew the remark, but not before the opposition erupted in anger, including shadow attorney-general Mark Dreyfus and MP Michael Danby, both Jewish.
"But it's silly to use an example of the ultimate evil in politics. He is the prime minister, he's supposed to have standards."
It was Abbott's second World War II jibe in recent weeks, after he sparked controversy for referring to a "holocaust" of job losses as he faced parliamentary questions over the government's economic plans.
He later apologised and withdrew the remark after Dreyfus said: "Comments like this should have no place in Australian political debate and should never have been made in the first place.