Newspapers quoting anonymous officials said a process was initiated to remove Telecom Minister Abdul Latif Siddiqui for allegedly making the comments at a rally of Bangladeshis in New York two days ago.
"The prime minister is very annoyed with his comments and ordered initiatives for his removal," a senior official said.
He added an official announcement was expected to be made after the return of President Abdul Hamid, who is now in Saudi Arabia to perform the Haj, and Hasina, who is on her way back after attending the UN General Assembly in New York.
Main opposition Bangladesh Nationalist Party (BNP) also denounced the comments.
Also Read
Bangladeshi television channels aired Siddiqui's comments over the Haj pilgrimage.
At the same function, Siddiqui also castigated the premier's son and her IT Advisor Sajeeb Wazed Joy.
Siddiqui, who is now in Mexico, told BBC that he would not withdraw his comments and that "there is nothing to regret".