The 53-year-old man allegedly followed food suppliers into the security perimeter and inside a restaurant at the village on Monday before shouting at players who were there for lunch, police inspector Song Myeong-su said today.
Song said North Korean judo representatives were among the athletes at the restaurant, but added that the verbal outburst didn't specifically target the North Koreans.
Song said the man has no known links with any anti-Pyongyang civic group.
Police have allowed the man to return home and haven't decided whether or not to press charges, Song said.