The 51-year-old actor paid tribute to Loretta on Instagram by sharing a picture of them together when he was baby, reported People magazine.
"The love of my life passed away yesterday in her home surrounded by her children and grandchildren," he captioned the photo.
"My mother had enough love to fuel a small country. She was truly one of a kind and will live forever in the hearts of all who knew her.
"A few weeks ago, I asked my mom if she remembered an inconsequential event - she simply replied, 'I only remember the love.' #Brokenhearted #Iloveyouthemost," he added.
Loretta was inspired to start the charity after her brother was diagnosed with terminal cancer many years ago.