He was accompanied by his Japanese counterpart Shinzo Abe when he went around the famous temple.
He was at the temple complex for about half an hour during which he enquired about the history of the eighth-century Buddhist pagoda.
The Chief Monk Mori took the Prime Minister around the complex having wooden temple including the five-storied main pagoda.
It stands 57 meters tall, making it the tallest pagoda in Japan, and has become a symbol of both the temple and Kyoto as it can be seen from many places across the city.
When Modi arrived at the temple, a number of Indians carrying the Tricolour were there to see him. He went and shook hands with the excited crowd.
Toji Temple is one of Kyoto's many UNESCO world heritage sites.