Recalling the incident, the production stopped because Spacey had been accused of sexually inappropriate behaviour towards a younger actor.
"I did not know honestly then the extent of his violence," Byrne told The Sunday Times.
"I mean, he was kind of a joke in that people would say, 'That's Kevin,' but nobody really understood the depth of his predations. It was only years later that we began to understand that (filming) was closed down for a particular reason and that was because of inappropriate sexual behaviour by Spacey," he added.
In the interview, Byrne also recalled working with disgraced Hollywood producer Harvey Weinstein, who is himself facing multiple allegations of sexual harassment.
More From This Section
"I did three movies with Harvey Weinstein, and I knew he was a sleaze-bag. I knew he was a vile bully and I saw his bullying up close. I saw him be absolutely appalling, not just to women but to men as well. He had very little respect for any kind of human being. He wanted his stars around him," Byrne said.
Following Rapp's statement, the "House of Cards" star issued an apology and also came out as a gay man, saying he does not remember the incident and that it "would have been deeply inappropriate drunken behaviour".