Based on 275 estimates from across the continent, a report by the conservation group put Africa's total elephant population at around 415,000, a decline of around 111,000 over the past decade.
It is the first time in 25 years that the group's African Elephant Status Report has reported a continental decline in numbers, with the IUCN attributing the losses in large part to a sharp rise in poaching.
"The surge in poaching for ivory that began approximately a decade ago - the worst that Africa has experienced since the 1970s and 1980s - has been the main driver of the decline," said IUCN in a statement.
IUCN chief Inger Andersen said the numbers showed "the truly alarming plight of the majestic elephant".
More From This Section
"It is shocking but not surprising that poaching has taken such a dramatic toll on this iconic species," she said.
The IUCN report was released at the world's biggest conference on the international wildlife trade, taking place in Johannesburg.
Thousands of conservationists and government officials are seeking to thrash out international trade regulations aimed at protecting different species.
A booming illegal wildlife trade has put huge pressure on an existing treaty signed by more than 180 countries - the Convention on International Trade in Endangered Species (CITES).