The 37-year-old musician took to Instagram where he posted his child's picture who appeared to be in a wondering daze.
"He's a happy little dude in the morning!! Love this little buddy," Kelly wrote, along side the photo.
Before Joshua, Kelley and wife Katherine Heigl were already parents to adopted daughters Nancy Leigh, 8, and Adalaide, 4.
Heigl has said she would love to have "more children".