"I am extremely angry about these heinous and despicable terrorist acts. We will never forgive terrorists," Abe told reporters in Tokyo.
"We will cooperate with the international community to make them atone for their crimes."
The claim was made in a video released online that included no mention of a Jordanian pilot also being held by IS, whom the jihadist group has also threatened to kill.
Goto, a journalist, is seen kneeling next to a standing masked man who speaks with a British accent and blames the Japanese government for his "slaughter".
Also Read
Yesterday, Japanese media quoted Deputy Foreign Minister Yasuhide Nakayama as saying that negotiations with the militant group were "deadlocked".
"The government has been working with the utmost efforts on the issue -- I'm deeply regret that this is the result," Abe said.
"Japan will never yield to terrorism...(and) is firmly resolved to fullfil its responsibility in the international community's fight against terrorism.
"I will take thorough measures to ensure the safety of Japanese citizens inside and outside Japan."
The premier was holding an emergency meeting after the brief press conference.