Kardashian, 28, is one of the big names in the running to replace Steve Jones as the host of the pop star search show.
"They're doing the auditions without a host so it's no big deal they don't have hosts right now. Simon really wants to take his time. The two person thing didn't work out the first time and he wants to make sure it works this time," a source said.
Nicole Scherzinger initially hosted the show alongside Jones, but then the 'Poison' singer took Cheryl Cole's place on the judging panel after the British singer was asked to leave the show.
However, Scherzinger and fellow panelist Paula Abdul did not return for the second series and were replaced by Britney Spears and Demi Lovato.