In a lengthy report, the New York-based body studied the cases of 189 people locked up in Karaj, 50 kilometres west of the capital Tehran.
HRW said that in 63 cases prisoners had been arrested, convicted and sentenced "solely because they exercised fundamental rights such as free speech and rights to peaceful assembly or association".
In 35 other cases, in which prisoners had been sentenced to death for terrorism-related offences, the group said it suspected "egregious" due process violations that may have tainted the judicial process.
"The lion's share of responsibility for releasing these prisoners rests with the judiciary, but President Hassan Rouhani and his cabinet should be doing more to press for their release," he added in a statement.
More From This Section
HRW said several dozen members of the Baha'i community, Iran's largest non-Muslim minority, were serving sentences for charges including "spreading corruption on earth".
The body also said two Christian pastors and two Christian converts were being held, as were nine journalists and bloggers and seven human rights lawyers including Mohammad Seifizadeh who is close to Nobel Peace laureate Shirin Ebadi.
Last month, three journalists - including the Washington Post's Tehran correspondent - were arrested. Yesterday, an official said they were being held for security reasons.
Observers say they fear the journalists' arrests could undermine the thorny negotiations between Iran and the West for a comprehensive nuclear deal that world powers are seeking to reach by November 24.