At least four people were confirmed dead and 55 injured.
The high number of people unaccounted for likely trapped in the ship or floating in the ocean led to fears of a drastic rise in the death toll, making it one of South Korea's biggest ferry mishaps since 1993, when 292 people died.
One student, Lim Hyung-min, told broadcaster YTN after being rescued that he and other students jumped into the ocean wearing life jackets and then swam to a nearby rescue boat.
Local television stations broadcast live pictures of the ship, Sewol, listing to its side and slowly sinking as passengers jumped out or were winched up by helicopters. At least 87 vessels and 18 aircraft surrounded the stricken ship.
Also Read
Those rescued wet, stunned and many without shoes were brought to nearby Jindo Island, where medical teams wrapped them in pink blankets and checked them for injuries before settling them down on the floor of a cavernous gymnasium hall.
The ship had set sail from Incheon, a city in South Korea's northwest and the site of the country's main international airport, last night for an overnight, 14-hour journey to the tourist island of Jeju.
Kang Byung-kyu, a government minister, said two of the dead were a female crew member and a male high school student.
He said a third body was also believed to be that of a student. A coast guard officer confirmed a fourth fatality but had no immediate details about it.
Kang said 164 people were rescued, of whom 55 were injured. Officials said 292 people were missing.