Gokhale, a 1981 batch officer of the Indian Foreign Service (IFS), was India's envoy to China.
He is at present secretary (economic relations) in the Ministry of External Affairs.
The Appointments Committee of the Cabinet has approved Gokhale's appointment to the post, said the order issued by the Department of Personnel and Training.
Jaishankar was appointed foreign secretary on January 29, 2015, for two years. He was given a one-year extension in January last year.