The accord signed by officials from both countries in Vienna provides for the restitution by the end of the year of 977 books currently sitting in Russian libraries.
"These books hold huge cultural significance for us," Austrian Foreign Minister Michael Spindelegger said at a news conference.
"I am delighted that, 70 years later, they will be returned to the Esterhazy castle."
The book collection, spanning four centuries, belonged to the House of Esterhazy, a Hungarian noble family based in what is now eastern Austria.