The former Conservative MP was Home Secretary in Margaret Thatcher's government from 1983 to 1985.
First elected in 1974, Brittan later became Trade and Industry Secretary and stood down from the House of Commons when he became a European commissioner in 1989.
His family paid tribute to his "extraordinary commitment to British public life".
As an MP, he represented the constituencies of Cleveland and Whitby and Richmond in Yorkshire.
Commons Leader William Hague, the current Richmond MP, said Brittan had been a "kind, assiduous and brilliant man".