The 41-year-old has been married to Affleck since 2005 and the couple has two daughters and a son together, reported Showbizspy.
"Jen really doesn't trust Ben. She's going to Cleveland for two weeks this month to begin shooting a new film. While she's away, she's pulling the kids out of school in Brentwood to be on set with her. She's worried about Ben being by himself," a source said.
"There was talk that Ben strayed with Blake Lively (during filming of The Town). And he started talking to Jennifer Lopez again. His wife checks up on him all the time and looks through his phone
Affleck is known to have said 'marriage is hard work' in his Oscar acceptance speech this year.