The 54-year-old is a judge on the singing contest, but he could be planning to leave the show since it is pulling in fewer viewers than in previous years, reported Contactmusic.
"People have talked as if it is all over for 'The X Factor' USA but that's not the case at all. It is true that ratings are down but it still does well with younger viewers if it was to come back next year it would have to have a lower budget.
The 'X Factor' USA is currently in its third season with the winner awarded a USD 1 million recording contract with Cowell's label Syco, however, in the previous two series' it was worth USD 5 million.