Swaraj, who was briefing the reporters on the arrangements to mark the Day, said "he (Modi) will grace the occasion and address the gathering but he will not perform yoga."
She was asked if the Prime Minister will perform yoga at the mega event on Rajpath which is expected to be attended by 35,000 people.
Modi, an avid practitioner of yoga for years, has been tweeting various 'asanas' and talking about their benefits in the run up to the International Yoga Day.
Describing yoga as the "soft power" of India, Swaraj said "the initiative India had taken there was a reason behind this...That the world which is now engulfed in violence should be taken to a path of peace.
"Yoga is India's soft power. Through that soft power, our goal is the world attains peace also bring down violence in the world.