The IOC stripped Nuzman of his honorary membership and removed him from the coordination committee for the 2020 Tokyo Olympics.
It also suspended Brazil's Olympic Committee (BOC), although it said the country's athletes were unaffected and would be allowed to take part in next year's Winter Olympics in Pyeongchang and will continue to receive scholarship funds.
The BOC responded by calling a general assembly on Wednesday to "consider the decisions announced by the IOC".
Nuzman, 75, was arrested in Rio de Janeiro on Thursday as part of a probe into allegations he helped buy votes from the IOC to secure Rio's hosting of the 2016 Games.
More From This Section
Brazilian officials said last month Nuzman was the "lynchpin" in a plot to bribe the IOC into awarding Rio the Games.
At the time, Nuzman was detained and questioned and authorities confiscated his passport.
Authorities allege that former Rio governor Sergio Cabral, who is serving a 14-year prison term for bribery and money laundering, masterminded the plot.
Investigators believe $2 million (1.7m euros) in bribes was paid to Papa Massata Diack, the son of Senegalese IOC member and former world athletics chief Lamine Diack, before the 2009 vote.
Rio beat bids from Chicago, Madrid and Tokyo to host the 2016 Games.
The Rio 2016 committee's chief operating officer Leonardo Gryner was arrested in a separate raid in Rio on Thursday.
Prosecutors say Nuzman's assets increased 457 percent over the past decade with no clear source for the new income.
Nuzman also tried to conceal his wealth, often hiding it in foreign accounts, the statement read.
He only declared he owned 16 gold bars weighing one kilogramme (2.2 pounds) each when he was questioned a month ago, prosecutors said.
The office of Switzerland's attorney general said on Friday it was "currently analysing" a request from Brazil for legal assistance in the case.
In a statement, the IOC said that "to protect the interests of the Brazilian athletes" the decision had no bearing on the country's teams.
"Therefore, the IOC will accept a Brazilian Olympic Team in the Olympic Winter Games Pyeongchang 2018," it said.
Rio 2016 was credited with being a sporting and organisational success, but revelations of massive corruption during the preparations and now even in the awarding of the Games have tarnished the legacy.