"(We must) make terrorists become like rats scurrying across a street, with everybody shouting 'beat them!'" Xi was quoted as saying by state-run Xinhua news agency.
His remarks at a study session on national security and social stability of the ruling Communist Party of China (CPC) came in the backdrop of a series of attacks by militants in the restive Xinjiang province, including the deadly attack at the Kunming railway station last month in which 33 people were killed and 143 others injured.
Calling terrorism the common enemy of the people, Xi urged the improving of counter-terrorism systems and abilities.
He also asked the public to build a "wall of bronze and iron" to fight terrorism.
Also Read
China has formed the National Security Council (NSC), integrating a number of security and intelligence agencies to offer a quick response to terror attacks and has deployed armed police in all the major cities.
Xi, who is the President, head of the CPC and the military, has also taken over as the chief of the NSC.
Violent terrorists ignore basic human rights, trample humanism and justice and challenge the bottom-line of human civilisation, Xi said.
It is neither an issue of nationality nor one of religion, but it is the common enemy of people of all nationalities, he said.
Xi emphasised on the fight against secessionism and promotion of ethnic unity and common prosperity. He called on a resolute strike on secession, infiltration and sabotage by hostile forces within and outside China.
To implement the overall national security outlook, China must attach importance to both external and internal security, homeland security and the safety of its people as well as traditional and non-traditional security, he said.
To safeguard national security, China must promote development in a more comprehensive, coordinated and sustainable way and improve people's livelihood so as to eliminate sources of social conflicts, Xi said.
He also called on perfecting systems to protect people's legal rights and the rule of law so that social conflicts can be settled through legal means effectively.