The Italian real estate developer began dating Hathaway, 29, in 2004 but his arrest put a strain on their relationship as the actress ignored his desperate attempts to contact her from prison, reported People magazine.
Follieri has now revealed that he knew that his relationship with Hathaway will end once the claims hit the news as the actress cut all ties to protect her good name in Hollywood.
"We were fighting a lot. The attacks in the media were putting her in a very difficult situation with her career. We talked about when we were going to see each other again. There was a planned party for me in Capri. She didn't tell me if she was coming or not. She (just) said, 'I'll love you forever'," said Follieri.
The 33-year-old was later convicted of his crimes and sentenced to three years in jail. He admitted that he knew he lost Hathaway forever when she wouldn't return his calls from behind bars.
He adds, "It was painful, but after, you accept."
The businessman has since been released and deported to his native Italy while Hathaway got engaged to actor Adam Shulman.