The man told the police that he left North Korea by ship last week, public broadcaster NHK and the Asahi Shimbun said, without clarifying why he was on board.
He then jumped off the boat and swam ashore in Yamaguchi prefecture by holding onto a floating plastic container, the news reports said.
The man, whose name was withheld, was found wandering in Nagato city yesterday morning and taken in by police, they added.
After questioning, the police will hand him over to immigration authorities to determine whether he is a North Korean defector who needs protection and assistance, Kyodo said.
Also Read
A local police spokesman declined to confirm the reports.
Nine North Koreans were picked up by Japan's coastguard in 2011 after spending five days at sea. They then travelled to South Korea to settle there.
Nearly 30,000 North Koreans have fled poverty and repression at home to settle in the capitalist South.