The attack in the eastern province of Kunar was the single deadliest strike on Afghan troops in recent months by the insurgents, and caused Karzai to cancel a planned two-day visit to Sri Lanka.
Afghanistan and Pakistan regularly accuse each other of supporting militant attacks in the other country. Karzai's comments came days after Islamabad said 23 of its kidnapped paramilitary soldiers had been killed on Afghan soil.
"During this insurgents attack on Afghan army checkpoint...21 Afghan national army soldiers were martyred, three wounded," defence ministry spokesman General Zahir Azimi said in a statement.
Reinforcements were sent to the area to hunt for the attackers and free the kidnapped troops, the defence ministry said.
Also Read
The rebels attacked the reinforcements with a suicide bomb and gun attack but caused no casualties, it added.
A senior official at Afghanistan's interior ministry who declined to be named told AFP more than a hundred militants "coming from the other side of the border" were involved in the operation.
He also said the authorities were investigating if this attack was linked to the killing of the Pakistani soldiers.
Karzai demanded action from Pakistan.
Karzai cancelled a trip to Sri Lanka where he had been due to hold talks with his counterpart Mahinda Rajapakse, the Sri Lankan foreign ministry said in Colombo.
Kunar provincial governor Shujah-Ul Mulk Jalala told AFP some of the soldiers in the post were thought to have aided the Taliban.