The collision happened near the island of Izu Oshima, about 120km (74 miles) south of the capital Tokyo.
Acoast guard official told AFP news agency: "There is nothing we can see around the waters. We found an empty raft. We are also seeing some life jackets on the water surface."
"But we are yet to find any parts of the vessel."
The missing crew includes seven South Koreans and nine Indonesians, from the South Korea-registered Orchid Pia.
It collided with the Panama-registered Cygnus Ace in rough seas early on Tuesday, when visibility was very poor because of rain.
Television footage has shown a wide oil slick in the area.
The South Korean ship was carrying steel parts to South Korea while the Panamanian ship was carrying vehicles.