Rahim made a fluent 77 while Mahmudullah remained unbeaten on his career best 82 as the duo helped Bangladesh recover from 32-4 to post a commendable 256-8.
Shakib Al Hasan, Jubair Hossain and Rubel Hossain then claimed two wickets each to restrict Zimbabwe to 235-8 and give Bangladesh a 4-0 lead in the five-match series.
Zimbabwe looked competitive for the first time in the series as medium pacers Neville Madziva (3-60) and Solomon Mire (3-49) put Bangladesh on the back foot with their opening spell burst.
Rahim led the initial charge before he top-edged leg-spinner Tafadzwa Kamungozi to end his 78-ball innings.
More From This Section
Skipper Mashrafe Mortaza hit 39 off 25 balls to ensure Bangladesh got a competitive score.
Zimbabwe made a brisk start in reply when Hamilton Masakadza (28) and Vusimuzi Sibanda (17) scored 48 runs in the opening stand.
Shakib removed both the openers in his successive overs and debutant leg-spinner Jubair bowled one-down Timycen Maruma to leave them struggling at 60-3.
Brendan Taylor and Mire each hit a fifty and put 106 runs for the fourth wicket to give them some hope but brought the hosts back into the contest again.
Shakib finished with 2-28 while Jubair and Rubel grabbed 2-42 and 2-48 respectively.
The fifth ODI will be held at the same venue on December 1.