He was 87.
Sharma, who was born in Dausa district of Rajasthan, served as the governor of Gujarat from 2004 to 2009.
He was admitted to a private hospital a few days ago following respiratory problems.
Sharma is survived by two sons. One of them, Brijkishore Sharma, is Education Minister in Rajasthan government.