Tundu Lissu, chief whip of the opposition in parliament and president of Tanzania's bar association, was arrested last week after holding a press conference where he criticised Magufuli and complained of a "climate of fear which reigns everywhere."
After appearing in court on Monday, Lissu was "returned to prison," ahead of a Thursday ruling on his application for bail, said Saed Kubenea, a member of parliament in Lissu's Chadema party.
Lissu faces a "hate speech" charge for his "insulting words that are likely to incite ethnic hatred," according to the charge sheet.
Lissu's comments and jailing come amid growing tension over the authoritarian leadership style of Magufuli, who is known by his Swahili nickname "Tingatinga" or "bulldozer".
He came to power in 2015 as a corruption-fighting "man of the people" but critics say there has been a rise in repression with opposition meetings banned, newspapers shut down and journalists and artists threatened for criticising his administration.
In his speech Lissu accused Magufuli of building "a system based on favouritism, nepotism, tribalism and regionalism.