Brokenshire resigned in a letter to Prime Minister Theresa May saying he needed surgery, on the same day the premier plans a wider reshuffle of her cabinet.
"I have a small lesion in my right lung which needs to be removed. Clearly, my long term health and my family are my priorities and I intend to proceed with surgery at the earliest opportunity," he wrote.
Brokenshire, 50, has hosted repeated unsuccessful negotiations between republicans Sinn Fein and the Democratic Unionist Party to form a power-sharing executive in British- ruled Northern Ireland.
The prime minister backed Brokenshire's decision to step down at "an important moment for politics in Northern Ireland", thanking him for his efforts with a letter in which she urged the parties to re-establish the power-sharing government.
More From This Section
The administration collapsed in January last year after Sinn Fein pulled out over a scandal involving a renewable energy funding scheme.
The government was set up as a result of the 1998 Good Friday agreement, which brought to an end three decades of mostly sectarian conflict in Northern Ireland that claimed the lives of 3,500 people.
Tensions have been raised further after the DUP struck a power deal with May's ruling Conservatives after a general election in June, in which she lost her parliamentary majority.
Sinn Fein argues that the deal gives the DUP unfair influence over British government policy in Northern Ireland, including over Brexit negotiations that will have major implications for Ireland's economy.