The assassination occurred on the very day that Serbia and Kosovo had resumed talks on normalising ties after a hiatus of more than a year.
The Serbian government official in charge of Kosovo, Marko Djuric, said the murder was "a criminal, terrorist act against the entire Serbian people".
Ivanovic was shot dead by gunmen firing from a car as he arrived at the headquarters of his party in the flashpoint town of Mitrovica at about 8:15 am local time, according to police.
He said Ivanovic, who was set to face a retrial on charges of war crimes over the Kosovo conflict after an earlier conviction was thrown out, had been hit by five bullets.
More From This Section
Police said they have found a burnt-out car that was presumably used in the attack, local media in Kosovo reported.
Public prosecutor Shyqri Syla told AFP that investigators were at the scene, but that it was not yet known who was behind the attack.
A former Serbian state secretary for Kosovo, Ivanovic was a key interlocutor with NATO, the United Nations and later the European Union after the 1990s war and was seen as backing dialogue with Kosovo's ethnic Albanians.
Serbian President Aleksandar Vucic swiftly called an emergency meeting of the Council for national security after the shooting, national broadcaster RTS reported.
"Whoever is behind this attack... whether these are Albanian, Serb or any other criminals, they have to be punished, they have to be brought to justice," Djuric said in Brussels, where he was heading the delegation due to hold talks with Kosovo Albanians today.
Under the pressure from the international community and European Union auspices, Kosovo and Serbia have been trying to normalise ties almost 20 years since the start of a bloody war that claimed 13,000 lives, mostly ethnic Albanians.
The 1998-99 war between Serbian security forces and Kosovo Albanian guerrillas was ended by a NATO air campaign.
Predominantly ethnic Albanian Kosovo unilaterally declared independence from Serbia in February 2008. But Belgrade has rejected the move and still considers the breakaway territory as its southern province.