The memory lapses are similar to those that occur in the early onset of dementia.
Ecstasy, also known by its chemical name MDMA, is a Class A drug.
Studies showed that even ten pills of the party drug a year can cause specific memory impairment.
The report published in journal 'Addiction', gives some of the first information available on the actual risk of using ecstasy.
As the nature of the impairments may not be immediately obvious to the user, it is possible people wouldn't get the signs that they are being damaged by drug use until it is too late.
According to the study, new ecstasy users who took ten or more ecstasy pills over their first year of use showed decreased function of their immediate and short-term memory compared with their pre-ecstasy performance.
Also Read
These findings are associated with damage of the hippocampus, the area of the brain that oversees memory function and navigation.
Interestingly, hippocampal damage is one of the first signs of Alzheimer's disease, resulting in memory loss and disorientation.
The study participants took an average of 32 pills each over the course of the year, or about two and a half pills per month.
Some participants took as few as ten pills over the year and still showed signs of memory impairments.
"By measuring the cognitive function of people with no history of ecstasy use and, one year later, identifying those who had used ecstasy at least ten times and remeasuring their performance, we have been able to start isolating the precise cognitive effects of this drug," said Lead author Dr Daniel Wagner.