Chisti said that he would also go to Lahore to meet Sarabjit Singh.
82-year-old Chisti is facing life sentence in a murder case and was on May 10 permitted by the Supreme Court to visit Pakistan for a temporary stay.
After receiving his passport from a local court in Ajmer, Chisti boarded Shatabdi Express along with his brother Jamil Chisti and others.
"I am happy that I am going to my home land to meet my family and friends. I would like to offer thanks giving prayer as soon as I reach airport and thereafter I will go to my home," Chisti told reporters after the train reached Jaipur railway station.
"Being a Muslim its my duty to thank God. I will meet family, children wife and my friends and those who come to see me. When I am in Pakistan, I would like to visit the whole of the country. I would like to go to Lahore and meet Sarabjit Singh," he said.
Sarabjit is an Indian national who is languishing in a Lahore jail in Pakistan for the last 22 years.
More From This Section
A PUCL activist who is accompanying him said that Chisti will go to Pakistan High commission in Delhi tomorrow and after completing remaining formalities, he will take a flight for Pakistan.
She said that his son Tariq is in Delhi and is waiting for him.
As per the apex court's direction, Chisti will have to deposit his passport at the Indian High Commission in Pakistan and return to India by November 1.
Chisti came to Ajmer in 1992 to visit his ailing mother. However, he allegedly got himself involved in a brawl in which one of his relatives was killed.
The apex court finally granted him bail on April 9 and since then he was living with his brother's family in Dargarh area in Ajmer.