The U-125 jet with six airmen on board left a military base in Kagoshima prefecture on the southwestern island of Kyushu on Wednesday afternoon, but later "disappeared from radar", according to the ministry.
The cause of the crash is still unknown, but Defence Minister Gen Nakatani told reporters the ministry would investigate.
Yesterday, the bodies of four crewmen were found as some 600 rescuers, including troops and fire fighters, searched for the plane near the base in bad weather.
Nakatani told reporters it was "extremely regrettable and sad that excellent crew members with a bright future ahead were involved in such an accident".