No one was injured in the incident, they said.
The fire broke out early morning on the ground floor and soon engulfed the first floor of the building, they said.
Large number of towels, which the factory used to manufacture, were gutted in the fire, they said.
Atleast six fire tenders were pressed into service and it took nearly four hours to control the fire, Chief Fire Officer Dinesh Verma said.
It is suspected that a short circuit caused the fire, he said, adding that efforts are on to ascertain the losses.