The sex of the baby is yet to be known, reported People magazine.
Kemper, who got married to the writer-creator in 2012, announced that she was expecting her first kid back in April during a taping of "The Tonight Show Starring Jimmy Fallon".
At the time, the 36-year-old actress told host Jimmy Fallon that she kept her pregnancy a secret while shooting the second season of her Netflix show.