"Our focus is Nawaz Sharif's government," Ansar ul Mujahideen said.
But "we will turn our guns on" Khan if polio campaign continues, Newsweek Pakistan reported.
The banned groups had earlier carried out attacks on anti-polio workers.
Cricketer-turned-politician Khan had earlier today appealed to the Pakistani Taliban and others not to attack polio workers and pledged to spearhead the vaccination campaign in the restive northwest that continues to be in the grip of the crippling disease.
Pakistan is one of only three countries in the world where polio remains endemic, and efforts to stamp it out have been hit by opposition from Islamist militants and attacks on vaccination teams.