The New York-based watchdog said rebel forces shot and killed two women in the main southern city of Aden last month and also unlawfully detained 10 local aid workers for up to two weeks.
"The incidents, possible war crimes, exemplify the grave threats to civilians in the embattled southern seaport," said HRW.
Its deputy Middle East and North Africa director Joe Stork warned that "the difficulty of investigating the fighting in Yemen may mean abuses like these in Aden are just the tip of the iceberg".
At least 38 civilians were killed Wednesday and 95 were wounded, including women and children, when shelling hit people trying to escape Aden by sea, according to a new toll given by city health chief Al-Khader Laswar.
More From This Section
Health officials and a spokesman for militia forces loyal to exiled President Abedrabbo Mansour Hadi blamed the rebels for the shelling.
Pro-Hadi forces, including military units and militia fighters, have been battling Iran-backed Huthi Shiite rebels and their allies for weeks in Aden.
"Aden's civilians are already in dire straits, without being attacked, detained, and held hostage," Stork said.
"Leaders of the Huthis and other forces need to protect civilians, not abuse and terrorise them."
The rebels have been battling to take control of Aden's Tawahi neighbourhood, which houses the headquarters of the pro-Hadi Aden TV.
Laswar said five more civilians were killed overnight in "random rebel shelling on their homes in Tawahi".
A military source meanwhile said that five Huthi rebels were killed in an ambush in Aden.