Demonstrators, holding placards reading "Murder" and "We want the truth", gathered outside the defence ministry two weeks after Hung Chung-chiu died of heatstroke apparently caused by being forced to exercise excessively.
Several sang songs mocking the military while others made a show of drinking bottled water -- a reference to the fact Hung's superiors reportedly refused to give him water despite his repeated requests.
"So many people come here today to demand truth," Hung's uncle, Hu Shih-ho, said.
Taiwan's defence ministry has apologised for Hung's death and military prosecutors are investigating the cause of his death.
More From This Section
"I apologise to you all," vice defence minister Andrew Yang told the crowd before bowing briefly to them.
"Please give the defence ministry a chance to correct previous mistakes and clarify the truth."
Organisers said more than 10,000 had turned out while police estimates were not immediately available.
Local media reported that Hung was at odds with his superiors for making complaints against them and that he had been placed in solitary confinement for taking a smartphone onto the army base.
Four military officials have been detained by authorities on abuse charges, while punitive measures have been placed on 26 other staff members.
The island's watchdog responsible for monitoring other parts of government The Control Yuan has announced an in-depth probe into alleged bullying in the military.