Under the new legislation, the entry for gender can be left blank on birth certificates, effectively creating a category for indeterminate sex in the public register.
But activists promoting the rights of so-called 'intersex' people said they hoped the creation of a third gender option would open the door to broader changes limiting genital surgery on newborns with both male and female characteristics.
"It's a first, important step in the right direction," Lucie Veith, an intersex person from the northern German city of Hamburg said.
"That we forbid cosmetic genital surgeries for newborns, that is our first demand," Veith said.
More From This Section
The German law is intended to remove pressure on parents to quickly make a decision about controversial sex assignment surgeries for newborns, but many advocates say it does not go far enough.
"The surgeries are likely to continue in Germany," said Silvan Agius, policy director at ILGA Europe, a lesbian, gay, bisexual, trans and intersex rights group.
"You can already say, 'No, thank you very much, I don't want any surgery until my child can choose his or her gender.'"
Operations on intersex babies and infants in many European countries take place without adequate informed consent by the patients, according to a 2012 European Commission report on the topic.
The report also found that many adults born intersex are angry these surgeries were performed without their consent.
Experts estimate one in 1,500 to 2,000 births result in a baby of indeterminate gender or both male and female gender features.