At the Hollywood Reporter's Producers Roundtable, the actor said he wanted Pitt to appear as Christian Grey in the movie, not himself.
'Fifty Shades' producer Michael De Luca joked that the role "looks a lot like Mark Wahlberg", to which Wahlberg replied, "No, sounds like Brad Pitt to me."
The 42-year-old spoke about how he almost purchased the rights to the film, before Universal & Focus Features eventually won the bidding war.
"I almost fired my agent Ari Emanuel over 'Fifty Shades' not because I wanted to play the part. We were aware of the book from very early on, and we were close to securing the rights, and then we get into this bidding war," he explained.
Also Read
"I wanted to produce because I was never going to get the script first. I had to wait for Brad Pitt to pass."
Jamie Dornan is now cast to play the elusive role of Christian Grey, alongside Dakota Johnson as female-lead Anastasia Steele.
The film will now open in cinemas on Valentine's Day weekend in 2015.