The couple welcomed little Jax Copeland Arnold, reported People magazine.
"We are still in a bit of shock. It has been a long journey and the healthy birth of our son is a miracle," Arnold said.
He then took to Twitter to share his happy news with his followers.
"Game Changer: Jax Copeland Arnold. 8 pounds. Looking forward to being a 70-year-old Little League Coach."
Arnold previously revealed he and his wife have been trying to start a family since they married in 2009, and they consulted fertility doctors in 2011 in their bid to conceive.
Groussman is Arnold's fourth wife. He was previously married to fellow comic, Roseanne Barr.