Subramani was trampled to death by the elephant late last night and his wife and sister suffered injuries, they said.
The body could be retrieved today, as the elephant was standing close to it till early hours, they said.
They were part of a group of pilgrims to the temple from Puliampatti in Erode district.
The injured have been admitted to hospital, police said.