Babar finished with 5-74 in only his third Test, while debutant Shah took 4-50 to dismiss a resolute Australia for 216 in their second innings and give Pakistan a 1-0 lead in the two-Test series.
Pakistan, who set Australia a 438-run target, were left frustrated as Mitchell Johnson (61) and Steven Smith (55) fought hard before they succumbed in a tantalising finish with just 21.5 overs left in the day.
Pakistan will now hope they either win or avoid defeat in the second Test in Abu Dhabi, starting October 30, to clinch their first series win over Australia since 1994.
Babar dismissed number ten Peter Siddle, caught bat-pad in the 92nd over, to seal the victory as Pakistani fielders removed the stumps even before the Australian review was turned down.
More From This Section
The defeat looked inevitable with Australia reeling at 117-7 at lunch but Smith and Johnson defied the spinners during their 65-run stand for the eighth wicket.
Babar threatened to dismiss Smith twice but Sarfarza missed a stumping with the batsman on 37 and Misbah-ul Haq failed to pick a difficult chance at 44.
Smith hit Shah for his third four to reach his seventh half-century but fell five runs later, caught at short-leg.
Smith resisted for 175 balls.
It was Babar who struck twice before lunch to bring Pakistan closer to victory.