President Salva Kiir yesterday said their release was the "price of peace", adding: "Even if we hang them they cannot compensate those who were killed."
The court order said the men were released "in order to promote peace and reconciliation among our people", while Kiir urged people to respect those released.
"This is the only way of getting out (of war)," he said in a speech. "Let's all work together."
"We were imprisoned without any reason," said freed detainee Pagan Amum, the former secretary general of the ruling party.
More From This Section
In a speech thanking his supporters, he vowed to work to end the vicious conflict.
"We have to return South Sudan to peace and stability," Amum said, adding he would work with both the government and rebels "to end this senseless war that is killing our people."
But Kiir has also ordered the four to remain in the country, warning that they can be "recalled back if the criminal procedures are reviewed".
US State Department spokeswoman Jen Psaki welcomed news of the releases as "a step that we support, but it should have happened long ago."
She warned the United States was "actively considering individuals" to be targeted for sanctions saying Washington continued "to be shocked and horrified by the violence on the ground."
Amid a wave of killings, UN human rights chief Navi Pillay is heading to the country on Monday on a joint mission with Adama Dieng, the UN's special envoy for the prevention of genocide.
Pillay also aims to meet with opposition leaders, either in South Sudan or in the Ethiopian capital Addis Ababa, where officials have been trying to inject life into stalled peace efforts.