Most of the estimated half a million Rohingya who have arrived in southeastern Bangladesh over the last five weeks after fleeing violence in Myanmar are crammed into the camps that have sprung up on government land.
But thousands of the mainly Muslim refugees have settled in the nearby district of Bandarban, part of the Chittagong Hill Tracts, where indigenous tribes waged a separatist insurgency in the 1980s and 1990s.
Bangladesh authorities fear their presence could revive communal tensions between the local Muslim population and the tribal minority, who are mainly Buddhist.
Banik said the government would begin moving them tomorrow to "ensure peace in the hill district".
More From This Section
Bangladesh has opened its borders to the Rohingya, who are denied citizenship in their native Myanmar.
But it has not granted them official refugee status and has made clear it does not want them to remain indefinitely.
Authorities have restricted the movement of the refugees, banning them from leaving the overcrowded camp areas where hundreds of thousands are living in desperate conditions with inadequate shelter.
Tribal groups ended their separatist insurgency in 1997 and signed a peace treaty with the government.
But tensions persist between the local Muslim population and the tribal groups, who have close ties with the ethnic Rakhine Buddhists accused of carrying out attacks on Rohingya in Myanmar.
In June this year local Muslims torched hundreds of houses in a tribal community following the alleged murder of a local politician.
And in May last year a 75-year-old Buddhist monk was found hacked to death in Bandarban, an attack later claimed by the Islamic State group.