The Philharmonic, famed for its annual New Year's concert, sent a letter to the descendants of the painting's former French owner Marcel Koch on Thursday to inform them of the move, a spokeswoman said.
The 1883 painting "Port-en-Bessin" by French artist Paul Signac, known for his pointillistic style, was confiscated in 1940 in France's Jura region and given to the Philharmonic after it performed in the region, the orchestra said in a statement.
"We are very happy that thanks to Sophie Lillie's efforts a breakthrough was finally achieved and we found the descendants," Philharmonic musician and president Clemens Hellsberg said in the statement.
"The restitution of this painting was of major concern to us. We have tried for years to deal with the Vienna Philharmonic's past and we want to face our responsibility in making up for past injustice."
Also Read
A ceremony will be held to return the painting to the family of Marcel Koch, but it was still unclear when this might occur.
Austria, which long considered itself a victim of Nazi Germany and only recently admitted shared responsibility in World War II and the Holocaust, has regularly been at the centre of the debate over the restitution of looted Nazi art.
In its most high-profile case in 2006, the state had to return Gustav Klimt's prized "Portrait of Adele Bloch-Bauer" to the family of the former owners after a years-long tug-of-war.
Last year, a Jewish family also laid claim to one of Austria's most famous artworks - Klimt's "Beethoven Frieze" - arguing the state had forced its sale after the war.