A fine Keisuke Honda strike after 16 minutes had put Japan in front at the Pernambuco Arena in Recife, but the Ivorians introduced Didier Drogba from the bench just after the hour mark before going on to turn the game on its head.
Within five minutes of the veteran striker's arrival on the rain-soaked surface, the Elephants scored twice, both times from Serge Aurier crosses.
The equaliser came when the right-back's curling delivery was headed home by Wilfried Bony, before Gervinho connected with another Aurier centre to stun the sizeable Japanese contingent inside the stadium.
The Elephants are desperate to finally progress beyond the group stage at a World Cup after failing in 2006 and 2010, and they were boosted in Recife by the presence in their starting line-up of Yaya Toure, who shrugged off a hamstring problem in time to captain the side.
However, what Japan lack in muscle they more than make up for in technical ability, and Honda displayed his class to put the Blue Samurai ahead.
Following a throw-in from the left flank, the ball arrived at the feet of AC Milan playmaker Honda at the edge of the box, and he took a touch before letting fly with a powerful, rising left-foot strike that gave Boubacar Barry Copa in goal no chance.