As a young boy, Modi had visited the Belur Math in Kolkata to join the Order, but his request was turned down and he was told that his calling was elsewhere.
Later, he got spiritual guidance from Swami Atmasthananda in Rajkot, Gujarat.
"The demise of Swami Atmasthananda ji is a personal loss for me. I lived with him during a very important period of my life," Modi wrote on Twitter.
He recalled that whenever he visited Kolkata, he made it a point to seek the blessings of the monk.
Swami Atmasthananda (98) today passed away at the Ramkrishna Mission Seva Pratishthan hospital in Kolkata.