The fire broke out due to explosion of crackers at the temple, a Home Ministry spokesperson said.
The spokesperson said 83 people were killed in the mishap and 350 others injured.
State police, state disaster management teams and fire brigade are on the spot for rescue and relief operations.
Union Home Minister Rajnath Singh spoke to Kerala Chief Minister Oommen Chandy and assured all central help to deal with the situation arising out of the tragedy.
"My heartfelt condolences to the families of those who died in the Kollam mishap. My prayers are with the injured," Singh said.