The incident occurred near Waghai town when the family was returning to Rajkot after their visit to the Shirdi Sai Baba temple, they said.
The four persons were killed on the spot after a weak trunk gave away and the tree fell on their moving car, police said.
The trunk had become weak from fires locals frequently lit to clear their farms, police said.
The deceased have been identified as Anurag Sharma (26), Shashibala Sharma (46), Tanya Sharma (18), and driver Mehul Chavda (55), they said.