Chasing 146 for win, India were looking down the barrel at 69 for five in nine overs. Stuart Binny (24) and debutant Sanju Samson (19) played cameos, but the visitors still fell short at the Harare Sports Club.
India, who had won the ODI series 3-0, thus ended with a 1-1 stalemate in the two-match Twenty20 series.
For Zimbabwe, Graeme Cremer snapped up three wickets for 18 runs in his four overs, while SC Williams (1/31), Taurai Muzarabani (1/23) and Chris Mpofu (1/26) took one each.
India suffered an early blow, losing skipper Ajinkya Rahane only in the third ball. He was run out after a mix-up with his partner Murali Vijay.
Also Read
Vijay (13) and Robin Uthappa (40) then added 53 runs in 34 balls before the former was sent packing by leg-spinner Cremer, who beat him for pace and toppled his off-stump in the seventh over.
Three balls later, Cremer struck again when he trapped new man Manish Pandey in front of the wicket with a googly that pitched on middle and leg and skidded on to beat the batsmen, leaving India at 57-3.
Jadhav too was run out when Muzarabani produced a direct hit as India lost half their side in nine overs.
Binny and Samson raised hopes with a 36-run partnership, but the former was sent back into the hut by a stunning catch by Chibhabha as Cremer picked up his third wicket.
In the 18th over, Samson holed out at long-on with Malcolm Waller taking the catch off Chris Mpofu as India slipped to 117-7.
Earlier, the Indians took wickets at regular intervals to deny momentum to the home side, but opener Chibhabha ensured that Zimbabwe put up a respectable total with a superb 51-ball 67 after his side elected to bat on a slow pitch.