During a landmark visit which has been overshadowed by Nelson Mandela's failing health, Zuma said Mandela and Obama were "bound by history" as the first black leaders of their respective nations.
"You both carry the dreams of millions of people in Africa," he said.
South Africa is the second leg of Obama's most substantial visit to Africa since becoming US president almost five years ago.
With China gaining influence on the continent, Obama -- whose father is Kenyan -- has been heavily criticised for not paying enough attention to Africa and waiting so long to visit.
More From This Section
But Obama's charm offensive has been overshadowed by the critical illness of Mandela, the former prisoner who became South Africa's first black president in 1994 after decades of apartheid rule.
"Our talks have taken place against the background of the ill health of our beloved former president Nelson Mandela, the founding president of our democracy," said Zuma.
"I know that he is your personal hero."
Obama said the long struggle against apartheid was a source of "personal inspiration".
He said Mandela's "moral courage" was an inspiration for the world. "The thoughts of Americans and people all round the world are with Nelson Mandela and his family."