US Olympics Committee (USOC) chief Scott Blackmun resigned citing health problems as the reason, according to various media reports.
Blackmum led the federation for more than eight years.
The 60-year-old was reportedly diagnosed with prostate cancer earlier this winter, and did not attend the Winter Olympics in Pyeongchang.
The resignation comes amid calls for his departure in accusation of not doing enough to react to sex abuse cases inside the US Gymnastics team, by former team doctor Larry Nassar.
In a statement, USOC chairman Larry Probst said, "Given Scott's current health situation, we have mutually agreed it is in the best interest of both Scott and the USOC that we identify new leadership so that we can immediately address the urgent initiatives ahead of us.
"The USOC is at a critical point in its history. The important work that Scott started needs to continue and will require especially vigorous attention in light of Larry Nassar's decades-long abuse of athletes affiliated with USA Gymnastics."
Nassar has been sentenced to up to 175 years for sexually abusing young gymnasts in his care.