Ballmer made his decision public a month after his successor, Satya Nadella, announced unprecedented job cuts at the US technology firm as it worked to adapt to a "mobile first, cloud first" world.
"Given my confidence and the multitude of new commitments I am taking on now, I think it would be impractical for me to continue to serve on the board, and it is best for me to move off," Ballmer said in a publicly posted letter to Nadella.
Nadella, in a responding letter posted online, thanked Ballmer for his contributions to Microsoft and said he supported the decision to leave the board.
Ballmer said he would be devoting time to new endeavors that include teaching and the Los Angeles Clippers NBA team he acquired earlier this month.
More From This Section
Ballmer paid a record USD 2 billion for the Clippers.
Microsoft last month announced its biggest job cuts ever as new chief executive Nadella called for a new focus at the US tech giant while integrating the Nokia phone division.
Nadella, who became CEO earlier this year, seeks to reinvigorate a company that had been the world's largest but which has lagged in recent years as Google and Apple have taken leadership of the tech sector.