Some 1,000 people, from war veterans to students, waved banners saying "China don't steal our oil" and "Silence is cowardly" -- a dig at Hanoi's handling of the dispute -- and sang patriotic songs in a park opposite the Chinese Embassy.
"This is the largest anti-Chinese demonstration I have ever seen in Hanoi," said war veteran Dang Quang Thang, 74.
"Our patience has limits. We are here to express the will of the Vietnamese people to defend our territory at all costs. We are ready to die to protect our nation," he told AFP.
The two countries are locked in long-standing territorial disputes in the South China Sea over the Paracel and Spratly islands, and often trade diplomatic barbs over oil exploration and fishing rights in the contested waters.
Also Read
Tensions between the communist neighbours have risen sharply since China unilaterally announced in early May it would move a deep-water drilling rig into disputed waters -- a move the United States has described as "provocative".
"I think that escalation is possible," analyst Nguyen Quang A told AFP.
Vietnam has alternated between tolerating anti-China rallies and violently breaking them up. The communist regime is wary of public gatherings that could threaten its authoritarian rule.
The leadership uses public protest as a means of expressing extreme discontent with Beijing, said Professor Jonathan London at City University of Hong Kong.
"Hanoi is well aware that permitting this kind of activity is a clear message to Beijing (and is also) keenly aware and anxious about maintaining social order," he said.
"Suppressing public protest against what has occurred would be regarded as absolutely illegitimate behaviour," he said.
Dozens of anti-China demonstrations have been held in Vietnam since 2007 to protest Beijing's perceived aggression over territory.