The 44-year-old singer is furious that his co-judges Mariah Carey and Nicki Minaj are at war, reported Showbizspy.
"Keith thinks that the producers don't pay him enough to to deal with 'this crap'," a source said.
Urban is said to be taking home around USD 4 million a season.
"Keith knew there would be friction between the two women, but had no idea how vicious and threatening it would become. What makes him uncomfortable is the level the animosity has escalated to, and where it could go from here.
And his wife, Nicole Kidman, is concerned as well," the source added.