Abe travelled with Prime Minister Tony Abbott to Rio Tinto's West Angelas mine, where he toured the site and observed the use of Japanese technology in state-of-the-art equipment.
Both Japan and China have competed for resources in the Asian region, with Australia benefiting from the energy-hungry countries as economic growth boosted demand.
Resource-poor Japan was one of Australia's first mining clients and remains a key buyer of its iron ore, the country's largest export, a point hammered home by both Abbott and Rio chief Sam Walsh.
"The Pilbara has been the source of the iron ore which has built so much of modern Japan and which has been the foundation of so much of modern Australia's prosperity.
More From This Section
"So it's very fitting and appropriate that we should go to the Pilbara to pay our respects."
Walsh, who greeted Abe at the mine, said his company's "iron ore business was born on the back of Japanese investment and we will never, ever forget this support".
"Japan is now one of Rio Tinto's most important trading partners and our enduring relationship for almost half a century symbolises the strengthening economic and trade ties between Australia and Japan," he said.
About 210.6 million tonnes, or 55 per cent of the total it imported between January and May, was sourced from Australia.
Japan is the second-largest importer, with Australian-sourced iron ore making up 61 per cent of all Japanese imports of the commodity in the first four months of this year.
"The Pilbara is strategic to China and Japan," said BBY senior resources analyst Mike Harrowell.
He added that the Abe visit was a symbolic gesture for both Japan and Rio, which late last year completed a major expansion of its iron ore operations in the Pilbara.
It follows the first shipment last month of liquefied natural gas to Japan from a US$19 billion project in PNG.