The Ayr Hospital surgeon was cutting into the pensioner's limb when the knife struck a metal plate in his leg.
After retail outlet of B&Q was found to be closed, the operation went ahead with the sterilised saw found in a storage area, the BBC reported today.
National Health Service Ayrshire and Arran said it was investigating an incident "where standard procedures were not followed".
A health board source said: "An elderly man who was a patient at Crosshouse Hospital needed a leg amputation and was taken to Ayr Hospital for the operation, because that's where the vascular surgeons are based.
Also Read
"That's when he discovered he'd hit a metal plate that they didn't know about. So he frantically sought advice from the consultant orthopaedic surgeon, who suggested going to B&Q."
However, the store was closed because the operation was being carried out after 9 pm, so the surgeon decided to use the saw which was from old hospital stock."
The source added: "The saw was sterilised by soaking in some disinfectant solution and the surgeon proceeded to complete the amputation after cutting through the metal plate.
It is understood the patient and his relatives were told about what happened some time after the incident.
Scottish Conservatives health spokesman Jackson Carlaw said: "This is simply incredible - an indescribable way to treat any patient.
"Despite the UK's advances in modern medicine this episode has all the finesse of improvised surgery on Nelson's flagship during the Battle of Trafalgar.
"I would hope that NHS Ayrshire and Arran thoroughly investigates this as a matter of urgency.