UNHCR spokesman Dan McNorton told reporters that at least 3,500 people -- mostly women and children -- had made it to the nearby town of Hasiya.
An emergency aid team that visited the area found families staying in three schools, an unfinished building, and in tents donated by local people.
"UNHCR was able to witness the dire humanitarian situation of these displaced families," said McNorton.
A further 3,000 refugees from Qusayr have been registered in neighbouring Lebanon, though the actual number fleeing is likely to be higher, he underlined.
More From This Section
Control of the border town is essential for the rebels as it is their principal transit point for weapons and fighters from Lebanon.
Lebanese Shiite militant group Hezbollah sent almost 1,700 fighters to Qusayr more than a week ago to support the Syrian regime's assault on the rebel stronghold, stoking fears that the country's civil war could spill across its borders.
Numbers snowballed as the conflict morphed into a sectarian civil war, and the total topped 1.6 million this week, said McNorton.
Most have fled to neighbouring Jordan, Lebanon, Turkey and Iraq.
In addition to the 1.6 million refugees, the UN says more than 4.25 million Syrians are displaced within Syria.
That means that, all told, over a quarter of Syria's pre-war population of 22.5 million have fled their homes since the conflict began.