Gibson was at the receiving end of for his anti-Semitic remarks while Baldwin has often landed in troble for his public rants.
Oldman slammed Hollywood for its inability to take a joke in a lengthy and candid interview to Playboy magazine's July/August issue while promoting his upcoming film 'Dawn of the Planet of the Apes', said the Hollywood Reporter.
The 56-year-old actor says he believes "political correctness is crap" and said people should get over a joke.
"I don't know about Mel. He got drunk and said a few things, but we've all said those things. We're all... hypocrites.
More From This Section
"Mel Gibson is in a town that's run by Jews and he said the wrong thing because he's actually bitten the hand that I guess has fed him - and doesn't need to feed him anymore because he's got enough dough. He's like an outcast, a leper, you know?," Oldman said.
'The Dark Knight Rises' actor also defended Baldwin's use homophobic terms during his frequent run-ins with paparazzi at a time when comedians get away with such words.
"At the Oscars, if you didn't vote for 12 Years a Slave you were a racist. You have to be very careful about what you say. I do have particular views and opinions that most of this town doesn't share, but it's not like I'm a fascist or a racist," Oldman said.