"IS advanced and took control of most of northern Palmyra, and there are fierce clashes happening now," said Rami Abdel Rahman, director of the Syrian Observatory for Human Rights.
He said 13 jihadist fighters were killed in ongoing clashes near the Islamic citadel in the city's west.
Abdel Rahman had no details on regime casualties.
Most of Palmyra's renowned ruins, including colonnaded streets and elaborately decorated tombs, lie to the southwest of the city.
More From This Section
The head of Syria's antiquities department, Mamoum Abdulkarim, meanwhile voice concern for the ancient site.
"I am living in a state of terror," Abdulkarim told AFP in a telephone call.
He said IS "will blow everything up. They will destroy everything," if the enter the site, adding that many of Palmyra's artefacts, like elaborate tombs, could not be moved.
"If they enter the ancient ruins, it will be worse than when Palmyra was defeated in the time of Zenobia," said Abdulkarim.
IS began its offensive on Palmyra on Wednesday and inched closer to the ancient metropolis on Thursday and Friday, executing at least 49 civilians over those two days according to the Observatory.