The coast was coated with rancid-smelling sticky white clumps of the oil as it washed in Sunday, with 11 beaches still closed to swimmers Monday in the height of a summer heatwave.
There are still lumps of the solidified oil on the beaches and the sea water in some areas is greasy.
Hong Kong comprises more than 200 islands with glittering bays, but there are increasing concerns about pollution and rubbish blighting its shores.
Although there is still a red flag up and the beach is officially closed, some people still ventured into the water.
More From This Section
One 61-year-old surfer, who gave his name as Simon and is a regular at the beach, said there was still oil in the water.
"It got under my feet and on my board. It's all slippery," he told AFP.
"Yesterday there was big chunks along the beach and in the water."
"I live here now, I have to put up with it. I don't like it," said Simon, who is an airport worker originally from Hawaii.
Beach announcements told determined swimmers at the closed beaches to get out of the water Monday.
But Agnes Mercado, 49, a regular at secluded South Bay on Hong Kong Island, was determined to take her morning dip, although she said she would not submerge her upper body.
"Of course I'm worried about it, but it's even worse than this on some days," she said of the pollution.
In a statement late yesterday it described the palm oil as "harmless to the human body".
But environmentalists still fear the potential impact and say the government has not done enough to contain the spillage.
"Whilst we may not see birds covered in black oil, palm oil is hazardous to wildlife in that it attracts bacteria," said Gary Stokes of the Sea Shepherd Conservation Society.
Stokes added that the oil will dissolve and break down, which could also remove oxygen from the water and kill marine life.
However, campaigners also say Hong Kong itself has a terrible track record on dumping of waste -- the city's landfills are groaning at capacity and there is no widespread recycling culture.