"Amidst the lived history of an extraordinary struggle for justice. Prime Minister met anti-apartheid revolutionaries and families," Ministry of External Affairs spokesperson Vikas Swarup said in a tweet.
"Reliving the life of two legends," Swarup said.
The Prime Minister also paid floral tributes to Gandhiji's bust and inaugurated a special exhibition featuring both Gandhi and Mandela.
"Homage to a remarkable history. PM visits Constitutional Hill, earlier a prison which housed both Gandhiji & Mandela," he said in another tweet.
Modi is on five-day four-nation tour. He visited Mozambique yesterday and is on the second leg of his tour of Africa here. He will also visit Tanzania and Kenya.