Twenty federal police agents swept into Rio de Janeiro's upscale Leblon neighborhood in an early morning operation to arrest 75-year-old Carlos Nuzman.
The Rio 2016 committee's chief operating officer Leonardo Gryner was arrested in a separate raid.
A grim-faced Nuzman left his home wearing a dark business suit in the Rio heat as he was escorted by police.
The agents, who were operating on orders from a federal judge, also seized documents.
More From This Section
Nuzman appeared relaxed and chatted with the agents as he entered the police station.
The men were sent to the Benefica prison in northern Rio, a police source told AFP.
The arrests are the latest in a widespread corruption scandal rocking Brazil that has ensnared many of the country's top political and business figures, and often has links to abroad.
It is "essential" that the suspects be detained "as a guarantee of public order" to allow their assets to be blocked, and to stop them from engaging in any criminal act or interfering with evidence, the Rio prosecutor's office said in a statement.
Nuzman, who has headed Brazil's Olympic Committee since 1995, saw his assets increase 457 per cent over the past decade with no clear source for the new income, prosecutors said.
Nuzman also tried to hide his wealth, often in foreign accounts, the statement read.
Following an investigation dubbed "Unfair Play" that spanned several countries, Brazilian officials last month said Nuzman was the "lynchpin" in a plot to bribe the International Olympic Committee into awarding Rio de Janeiro last year's Games.
At the time, Nuzman was detained and questioned and authorities confiscated his passport.
Today's operation in Rio was dubbed "Unfair Play - Second Half."
Authorities allege that former Rio governor Sergio Cabral, who is serving a 14-year prison term for bribery and money laundering, was the mastermind of the plot, which saw USD 2 million in bribes paid to Papa Massata Diack, the son of Senegalese IOC member Lamine Diack before the 2009 vote.