The destruction of the Baal Shamin temple, considered the second-most significant in ancient Palmyra, raised concerns for the rest of the UNESCO World Heritage sites.
It comes only days after IS beheaded the 82-year-old retired chief archaeologist of Palmyra, sparking widespread condemnation.
"This destruction is a new war crime and an immense loss for the Syrian people and for humanity," said Irina Bokova, the head of the UN cultural watchdog UNESCO, calling for the perpetrators to be held accountable.
Syria's antiquities chief Maamoun Abdulkarim told AFP the temple was destroyed yesterday.
More From This Section
"Our worst fears are sadly being realised," Abdulkarim said.
Famed for its well-preserved Greco-Roman ruins, Palmyra was seized from government forces in May, prompting concerns IS might destroy it as it has heritage sites in parts of Syria and Iraq under its control.
Initially most of Palmyra's best-known sites were left intact, though there were reports IS had mined them and the group reportedly destroyed a well-known statue of a lion outside the city's museum.
"The cella (inner area of the temple) was destroyed and the columns around collapsed," he said.
The Syrian Observatory for Human Rights, a Britain-based group that monitors the country's civil war, confirmed the destruction of the temple.
But the Observatory said Baal Shamin had been destroyed a month ago - a discrepancy which could not be immediately explained as information on Syria's civil war is often unclear.
IS captured Palmyra on May 21, sparking international concern about the fate of the heritage site described by UNESCO as of "outstanding universal value".
Known as the "Pearl of the Desert", Palmyra is an oasis town about 210 kilometres northeast of Damascus.