The UN health body outlined its stance in a statement late yesterday after a group of 150 international doctors, scientists and researchers sent an open letter outlining fears that the arrival of half a million tourists for the Games could cause the virus to spread more rapidly around the world.
But WHO said moving the Olympics would not have a major impact on the spread of Zika.
Zika can cause birth defects, including a devastating syndrome known as microcephaly in which babies are born with unusually small heads and brains.
Nearly 1,300 babies have been born in Brazil with the irreversible defect since the mosquito-borne Zika began circulating there last year.
Also Read
Addressed to the world health body, the letter said that pressing on with the Games in Rio, the second-most affected city in Brazil, would be "irresponsible" and "unethical".
"An unnecessary risk is posed when 500,000 foreign tourists from all countries attend the Games, potentially acquire that strain, and return home to places where it can become endemic," said experts from the United States, Britain, Canada, Norway, the Philippines, Japan, Brazil, South Africa, Turkey, and Lebanon, among others.
But the WHO statement said Brazil was "one of almost 60 countries and territories which to-date report continuing transmission of Zika by mosquitoes.
"People continue to travel between these countries and territories for a variety of reasons. The best way to reduce risk of disease is to follow public health travel advice," it said.
The Olympics and Paralympics will take place between August 5 and September 18, which is wintertime in Brazil, with the WHO noting earlier this month that it was the season when there were "fewer active mosquitoes and the risk of being bitten is lower".