The bodies of the Taiwanese skipper, three Chinese and one Indonesian crew members have already been recovered after the collision on Friday, and authorities have launched an investigation.
A coastguard spokesman said that rescuers were still searching for three more Indonesian and one Chinese crew listed as missing from the fishing boat.
The 48-tonne Shih Hui 31 was sailing off Chuwei, a port near Taoyuan city, early Friday when it collided with the 8,000-tonne cargo ship carrying gravel, authorities said.
"The duo were considered suspects after an initial investigation," Wang Yi-wen, spokesman for the Prosecutors' Office in the northern Taoyuan city, told AFP.
But he would not give further details, saying only that their testimony would be doublechecked against evidence on the boat such as radar-mapping data.