The American twins announced their withdrawal on Facebook yesterday, saying that as "husbands and fathers, our family's health is now our top priority."
They didn't mention the Zika virus, which has been linked to birth defects and been cited by several athletes in tennis and other sports who opted to skip the Summer Games that begin next week in Brazil.
The Bryans won a gold medal at the 2012 London Olympics, after collecting bronze in doubles at the 2008 Beijing Games. They have won 16 Grand Slam titles, a record in men's doubles.
The top-ranked U.S. Man in singles, John Isner, and Sam Querrey also are skipping the Rio Olympics. Isner said he is not going because ATP ranking points are not available.
Wimbledon runner-up Milos Raonic, Wimbledon semifinalist Tomas Berdych and Simona Halep are among the other tennis players who have pulled out of these Olympics. They each cited concerns about Zika.