The 'Argo' star says his fellow actors have now changed the way they treat him since he began working behind the camera and he is not let in on any secrets, reported Showbizspy.
"I'm not one of the guys any more. I remember I was walking over to the house guests and I heard one of the guys talking about getting stoned - I think somebody had brought something into the house - and as I arrived everybody clammed up.
"I thought, 'When did I become this guy? What am I, the dad?," he said.
Affleck, 40, who has three children - Violet, six, Seraphina, three, and eight-month-old Samuel - with wife Jennifer Garner, says he had forced the six actors who play hostages in his spy thriller "Argo" to live life without any luxury and they hated him for the experience.
"I took the six and made them live on the set for a week, gave them all the newspapers and the records and the magazines and their wardrobes, took away all their phones and closed the door.
"They were having bad iPod withdrawal. There was so much bitter complaint we had to have a production assistant standing outside with the phones in case of an emergency call," he said.