The raid early this morning happened in the town of Sid on the border with Croatia.
The migrants, who allegedly did not register with local authorities, were packed into buses and sent to southern Serbia and the border with Macedonia.
They are mostly from Afghanistan and Pakistan.
Only the elderly, women and children were allowed to remain in a local refugee center.
Sid Mayor Predrag Vukovic says the migrants "stole, fought among each other and attacked the locals."