The measure marks a triumph for activists after a decades-long struggle to clear the names of gay men who lived with a criminal record under Article 175 of the penal code.
An estimated 5,000 of those found guilty are still alive.
The legislation was passed by Chancellor Angela Merkel's cabinet and will soon head to parliament, where her ruling right-left coalition enjoys a large majority.
"Article 175 destroyed careers and ruined lives," Justice Minister Heiko Maas said in a statement. "The few victims who are still alive today deserve to finally have justice."
More From This Section
The legislation was named after World War II hero Alan Turing who was prosecuted under the law in 1952 and forced to undergo chemical castration treatment. He committed suicide two years later at the age of 41.
However the British measure, unlike Germany's, only automatically pardoned dead people while the living must still make an individual application to have their names cleared. It also failed to provide compensation.
Germany's Article 175 outlawed "sexual acts contrary to nature... Be it between people of the male gender or between people and animals".
Although the law dated from 1871, it was rarely enforced until the Nazis came to power, and in 1935 they toughened the legislation to carry a sentence of 10 years of forced labour.
More than 42,000 men were convicted during the Third Reich, and sent to prison or concentration camps.
In 2002, the government introduced a new law which overturned their convictions, and also applied to those convicted of desertion during Nazi rule.
But that move didn't include those convicted after the war when Article 175 was still in force.
It was finally dropped from the penal code in communist East Germany in 1968.
The compensation scheme under the new legislation includes a one-off payment of 3,000 euros (USD 3,200) for every man convicted and an additional 1,500 euros for each year spent in prison.