The Japanese-registered Eifuku Maru No 18 capsized after it crashed with the Sierra Leone-registered Jia Hui about 11 kilometres west of Izu Oshima island, coast guard spokesman Yoshiyuki Terakado said.
The coast guard sent patrol boats and helicopters to search for the crew. Five bodies were found inside the capsized boat late in the afternoon and their hearts were no longer beating, said Koji Takarabe, another coast guard spokesman.
The Jia Hui's 12 Chinese and one Malay crew were unharmed, and were being questioned aboard the ship about what had happened. It was still unclear what caused the collision, Takarabe said.