Obama himself made the surprise announcement at a White House media availability convened at a very short notice.
Hagel had tendered his resignation a few hours ago.
He has been asked to stay till a replacement is announced by Obama and is confirmed by the Senate.
"Over nearly two years, Chuck has been an exemplary defence secretary, providing a steady hand as we modernised our strategy and budget to meet long-term threats while still responding to immediate challenges like IS and Ebola," Obama said in his address to the media at the White House's State Dining Room with Vice President Joe Biden, Hagel and the top Pentagon leadership standing by his side.
Also Read
Robert Gates and Leon Panetta were the earlier two defense secretaries.
Obama said, Hagel has played a key role in the accomplishments of his administration in the defence sector.
"Today, our men and women in uniform are taking the fight against IS in Iraq and Syria, and Chuck helped build the international coalition to ensure that the world is meeting this threat together," he said.
Hagel said it was the right time to have a new leadership at the Pentagon.
"As the president noted, I have, today, submitted my resignation as secretary of defence. It's been the greatest privilege of my life to lead and most important, to serve with the men and women of the defence department and support their families. I am immensely proud of what we've accomplished during this time," he said.