Gabbay said that he was "unable to swallow" Prime Minister Benjamin Netanyahu's decision to take the defence portfolio from former general Moshe Yaalon and hand it to Avigdor Lieberman, who has pledged harsh measures against Palestinian "terrorists".
Yaalon resigned from the government a week ago in protest, warning of a rising tide of extremism in the party and the country as a whole.
"I could not accept the removal of Yaalon, a professional defence minister," Gabbay said.
"Recent political events and the changing of the defence minister are to me grave events which ignore what is important to state security and will cause further extremism in society."
Also Read
Gabbay, of the centre-right Kulanu party, is not a member of parliament and his resignation does not affect the ruling rightwing coalition's majority.
Co-opting Lieberman and his Yisrael Beitenu party will add five lawmakers to Netanyahu's previously wafer-thin majority if the coalition deal is given parliamentary approval next week as expected.
US State Department spokesman Mark Toner, in a rare comment on Israeli internal politics, said Wednesday that Washington had "seen reports from Israel describing it as the most right-wing coalition in Israel's history".
"And we also know that many of its ministers have said they oppose a two-state solution," he said.