Auden is 61-year-old Grammar's seventh child and his third with Kayte, who he married in 2011.
The couple are also parents to Kelsey Gabriel Elias, 2, and Faith Evangeline Elisa, 4.
Grammer also has four older children from previous relationships, daughters Spencer, 33, Greer, 24, and Mason Olivia, 15, as well as son Jude Gordon, 12.
Auden was born on Monday, November 14 and weighed 7 pounds and measured 21 inches long.
"Both mother and son are doing great," a representative for the couple told People magazine.
More From This Section
"The name was chosen to honour the great poet WH Auden, and James is after another great poet and singer James Taylor - two of our favorites," they said.
Ellis however was Grammer's maternal great-grandfather's name.
"He was a colorful character, to say the least, and we wanted to give the name a second chance," the couple added.
Grammer married Kayte, a former flight attendant from Britain, in 2011. It was the fourth marriage for the former "Cheers" actor.