Zaccheroni, 61, brought a squad full of optimism to Brazil but the Blue Samurai suffered a brutal reality check culminating in Tuesday's 4-1 rout by Colombia.
"I am disappointed and not satisfied with the results," the Italian told a press conference at the team's base camp in Itu, Brazil, according to Kyodo news agency.
"I really wanted to take us forward and qualify for the second round and I am so disappointed. But I picked the team, decided on the tactics and how we play and I want to take full responsibility."
Zaccheroni's contract was due to expire after the tournament but his resignation leaves Japan needing to bed in a new coach quickly before their Asian Cup defence in January.
Japan waltzed through qualifying and came to Brazil with high hopes and a squad including Manchester United's Shinji Kagawa, Keisuke Honda of AC Milan and Inter's Yuto Nagatomo.
But after taking the lead against Ivory Coast, they lost the opener 2-1 and then drew 0-0 against Greece. Tuesday's big defeat to Colombia left them rock-bottom in Group C.