Cowell, 53, is one of the creators of 'America's Got Talent' and enjoyed a romance with Lauren Silverman, who is reportedly pregnant with their first child, reported Contactmusic.
"I don't think Simon is thrilled. He is too busy with sex. If he was here doing his job and running things, there would be no baby," Stern said.
"I was wondering why he never shows up on the set, he's like an executive producer of the show, now I know he's busy making kids," he added.