"I told the president, 'Mr President, with all due respect you are making a big mistake,'" the top Democrat in the Senate, Chuck Schumer, told reporters.
The Senate Minority Leader demanded that the Department of Justice appoint a special prosecutor to look into allegations of Russian hacking and its influence on the 2016 elections.
"The American people need to have faith that an investigation as serious as this one is being conducted impartially, without a shred of bias. The only way the American people can have faith in this investigation is for it to be led by a fearless, independent special prosecutor," he said.
Schumer also questioned the timing of the shock decision, which came days after Comey testified on Capitol Hill about the FBI's investigation into Russia's election meddling and a possible collusion between the country and Trump's campaign.
More From This Section
"If the administration had objections to the way Director Comey handled the Clinton investigation, they had those objections the minute the president got into office. But they didn't fire him then. Why did it happen today?" he asked.
House Democratic Caucus Chairman Joe Crowley said Comey's dismissal is extremely troubling.
"President Trump fired the man investigating him and his cohorts. I strongly support calls for the appointment of a special prosecutor," he said.
Terming the move as "nearly unprecedented", Indian- American Congressman Raja Krishnamoorthi said it was extremely disturbing that the Chief Executive is interfering with an ongoing investigation into his administration by firing the person charged with conducting it.
Republican Senator John McCain, Chairman of Armed Services Committee said he is disappointed by Trump's decision to dismiss Comey.
"James Comey is a man of honor and integrity, and he has led the FBI well in extraordinary circumstances. I have long called for a special congressional committee to investigate Russia's interference in the 2016 election," he said.
"The president's decision to remove the FBI Director only confirms the need and the urgency of such a committee," McCain said.