The 55-year-old 'X Factor' boss and Silverman welcomed their first baby, son Eric, last year and have spoken of their desire for the boy to have siblings.
But Cowell believes adoption may be the couple's best option for a second child, reported Daily Star.
"Myself and Lauren want to have more children of our own but we could adopt as well. I wouldn't rule out it happening. It's a good idea. I'm beginning to warm to the idea of adopting," Cowell said.
Eric is Cowell's only child, while Silverman has an eight-year-old son with her ex-husband Andrew Silverman.