The 26-year-old left the couple's residence after Stewart confessed publicly to cheating on him with "Snow White and Hunstman" director Rupert Sanders, reported People magazine.
"I'm not sure they'll be able to recover from this. Robert is heartbroken and angry," a source said.
After pictures emerged of Stewart and Sanders, - who has two children with his wife Liberty Ross - kissing, the 22-year-old actress issued a statement saying she was sorry to have hurt Pattinson.
"This momentary indiscretion has jeopardized the most important thing in my life, the person I love and respect the most, Rob. I love him, I love him, I'm so sorry," Stewart had said after the incident became public.
The actress and Pattinson started dating after meeting on the set of the first "Twilight" film.