Indian players and fans had their hearts in their mouths as the the hosts needed only 10 runs off the last over but Zimbabwe skipper Elton Chigumbura's second One-day hundred was not enough to see the team through and they fell short by four runs chasing 256.
Ambati Rayudu produced a career-best unbeaten 124 and his second international century helped India recover from a top-order collapse and post a competitive 255 for six after they were inserted in by the hosts.
As expected, it was far from a cake walk for India as Zimbabwe showed stomach for a fight before being stopped at 251 for seven.
India's bowlers were far from impressive as they consistently erred in line and length, bolwing a lot of loose full tosses.
More From This Section
Like India, Zimbabwe also kept losing wickets intermittently but Chigumbura kept the African minnows in hunt with his fighting 104, that came off just 101 balls.
Along with Graeme Cremer (27) he raised a 86-run partnership for the seventh wicket which threatened to take the match away from India but the latter played an unnecessary reverse sweep to be out in the last ball of the penultimate over.
Chigumbara was left crestfallen as he could not steer the team past the finish line despite getting so close.
Left-arm spinner Axar Patel (2/41) and Stuart Binny (2/54) shared four wickets between them while Harbhajan Singh, Bhuvneshwar Kumar and Dhawal Kulkarni took one wicket apiece.
While Rayudu decorated his 133-ball knock with 12 fours and a solitary hit over the fence, Binny struck six boundaries and two sixes during his knock.
Apart from the duo, captain Ajinkya Rahane (34) was the only other Indian batsmen to register a double digit score.
Chamu Chibhabha (2/25) and Donald Tripano (2/48) picked up two wickets apiece for the hosts.