The actor breathed his last on Sunday in Mount Sinai St Luke's Hospital in New York, according to producer and family friend Anna Glass, reported Ace Showbiz.
Geoffrey, who was also dancer, choreographer, composer, designer and painter, started his movie career in 1962's British movie 'All Night Long'.
He later appeared in 1967's 'Doctor Dolittle' along with Rex Harrison, 1982's 'Annie' and 1992's 'Boomerang' with Eddie Murphy.
Geoffrey won two Tony Awards for Best Costume Design and Best Direction of a Musical in 'The Wiz', an all-black version of 'The Wizard of Oz', in 1975.
Most recently, Geoffrey provided narration for Tim Burton's 'Charlie and the Chocolate Factory'.
He is survived by his wife, Carmen de Lavallade, and their son, Leo.