Images of the rising flood waters in the River Dee show the 16th century Abergeldie Castle just a few feet from the water's edge, with parts of the estate reportedly being swept away.
John Gordon, the 21st Baron of Abergeldie, decided to evacuate his 450-year-old home with his wife when they heard a wall in their garden collapse into the river.
Concerns remained that the four-storey home, which is two miles from the Queen's Balmoral estate, could yet fall into the river.
He is now understood to have moved to another property on the estate, 'The Guardian' reported.
Also Read
The baron is a friend of the British monarch and rents out a portion of his 11,700-acre sporting estate to the Royal Family for shooting and fishing.
"He left the castle when the river was at its height. It swept the embankment away. It moved a 60ft lump of ground and took a lot of big mature trees as well," Fraser told the 'Scottish Daily Record'.
On Sunday, the Met Office issued an amber warning ("be prepared") for heavy rain for the Grampian and Central, Tayside and Fife areas until Monday night.