Twenty federal police agents swept into Rio de Janeiro's upscale Leblon neighborhood in an early morning operation to arrest 75-year-old Carlos Nuzman.
The Rio 2016 committee's chief operating officer Leonardo Gryner was arrested in a separate raid.
A grim-faced Nuzman left his home wearing a dark business suit in the Rio heat as he was escorted by police.
The agents, who were operating on orders from a federal judge, also seized documents.
Nuzman and Gryner "will be charged with the crimes of corruption, money laundering and criminal association," police said in a statement.
More From This Section
The men were sent to the Benefica prison in northern Rio, a police source told AFP.
With their detention, the vice president of Brazil's Olympic Committee, Paulo Wanderley, has temporarily taken charge of the body.
- Assets increased 457% -
=========================
The arrests are the latest in a widespread corruption scandal rocking Brazil that has ensnared many of the country's top political and business figures, and which often has links to abroad.
It is "essential" that the suspects be detained "as a guarantee of public order" to allow their assets to be blocked, and to stop them from engaging in any criminal act or interfering with evidence, the Rio prosecutor's office said in a statement.
Nuzman, who has headed Brazil's Olympic Committee since 1995, saw his assets increase 457 percent over the past decade with no clear source for the new income, prosecutors said.
Nuzman also tried to hide his wealth, often in foreign accounts, the statement read.
He only declared he owned 16 gold bars weighing one kilogram (2.2 pounds) each when he was questioned a month ago, prosecutors said.
- 'Unfair Play - Second Half' -
===============================
Following an investigation dubbed "Unfair Play" that spanned several countries, Brazilian officials last month said Nuzman was the "lynchpin" in a plot to bribe the International Olympic Committee into awarding Rio de Janeiro last year's Games.
Investigators also searched the offices of Brazil's Olympic Committee and several companies suspected of links to the scandal.
Thursday's operation in Rio was dubbed "Unfair Play - Second Half."
Authorities allege that former Rio governor Sergio Cabral, who is serving a 14-year prison term for bribery and money laundering, was the mastermind of the plot, which saw $2 million in bribes paid to Papa Massata Diack, the son of Senegalese IOC member Lamine Diack, before the 2009 vote.
Businessman Arturo Soares, known as "King Arthur," a top contractor for Cabral's administration, allegedly delivered the payment three days before the IOC vote in Copenhagen in October 2009. Rio beat bids from Chicago, Madrid and Tokyo to host the 2016 games.
=======================
Rio prosecutors say they have new information conclusively showing how the payments were made. In their statement they cite an email from Papa Massata directly asking Nuzman for help with "the final process," and an email discussing difficulties in bank transfers.
Rio 2016 was credited with being a sporting and organizational success, but revelations of massive corruption during the preparations and now even in the awarding of the Games have tarnished the legacy.
The latest scandal cast a pall over an IOC meeting in Lima last month following the designation of Paris for the 2024 Games and Los Angeles as the 2028 host city.
Ban Ki-moon, chief of the IOC's ethics commission, on Thursday also pledged cooperation and asked Brazilian authorities to supply all the available information on the case.
"Given the new facts, the IOC Ethics Commission may consider provisional measures while respecting Mr Nuzman's right to be heard," the IOC said in a statement.
The group "will not comment further on this matter until a recommendation is issued by the IOC Ethics Commission," and it "reiterates that the presumption of innocence prevails.