Family spokesman James Garcia said Castro died in his sleep in San Diego, where he was in hospice care.
Castro was a self-made man, the embodiment of the American dream. Born in Mexico, he overcame poverty and discrimination to graduate from university and launch a successful career in politics and diplomacy.
"America is the land of opportunity," Castro told The Associated Press in 2010. "Here, one can accomplish whatever they want to be. But you've got to work for it."
Growing up on the US-Mexico border near Douglas, Arizona, Castro saw discrimination around him. He said he wondered why the Hispanics were laborers and none delivered the mail or worked in offices.
He set out to beat the odds. When he couldn't get a job as a teacher schools didn't hire educators of Mexican descent back then he became a drifter for a while, working as a farm hand and boxing here and there.