It was not immediately clear why President Yahya Jammeh's government had taken this step, but Washington has criticized its rights record.
A State Department official in Washington told AFP the United States had "registered concern" with Gambia over the withdrawal of the officers.
"We will continue to closely monitor and assess the situation as events unfold," he said.
"The US embassy sent a security message to citizens to alert them to the embassy closure."
An AFP reporter in Banjul who visited the US mission confirmed that the Police Intervention Unit officers normally deployed there were gone.
More From This Section
Fanta Darboe Jawara, a 45-year-old naturalized US citizen from Frederick, Maryland, was held on April 16 after an opposition protest.
Her family told US media she was not involved in the protest and was simply waiting for a taxi when she was arrested and beaten by police.
But the president of Gambia's opposition UDP party told AFP that Jawara had been detained at the home of human rights lawyer Ousainou Darboe.
Darboe's house is a short distance from the US embassy, UDP president Dembo Bojang said.
A State Department official would not speculate on whether the police withdrawal was connected to controversy over Jawara's arrest.
"We will continue to monitor her case and provide all possible consular assistance," he said.
Jammeh seized power in a bloodless coup in 1994 and is regularly accused of a catalogue of rights abuses.
He recently told UN chief Ban Ki-moon and Amnesty International to "go to hell" for urging an inquiry into the death of a protestor in custody.