Murthy, who took the oath on the Gita, is now the highest ranking Indian-American in the Obama Administration.
"To have the opportunity to serve as Surgeon General is an extraordinary honour and a profound responsibility, and I want to thank President Obama for entrusting me with the stewardship of this office," he said in his address on the occasion at Fort Myer military base.
Murthy is the country's 19th Surgeon General.
In addition to practicing internal medicine at Brigham and Women's Hospital, he in 2009 started a nonprofit, Doctors for America, composed of thousands of doctors supporting President Barack Obama's healthcare initiatives.
Also Read
As the US Surgeon General, Murthy carries the rank of Vice Admiral. The Health Secretary Sylvia Burwell also participated in the event.
"He is eminently qualified for the job," Biden said on the occasion and described his family as an "incredible" in particular his grandmother who was also present.
"My family was never supposed to have left our ancestral village. My father is the son of a farmer in rural India. He was supposed to have been a farmer, as was I. But for my grandfather's insistence that his son get an education - even if that meant going into debt - we might have never left that village to go out in the world and - as my grandfather also insisted - start fixing what needed fixing," he said.
In his first major policy speech, Murthy vowed to improve the public health of the country.
"Public health does not exist in a vacuum. It is intrinsically linked to education, employment, the environment and our economy. There is a whole world beyond hospital corridors and clinic waiting rooms where people are struggling with issues of transportation, housing and development," he said.