The student's union at the Bradford University in northern England will vote on a motion calling for Khan to step down later this month.
Mohsin Tanveer, a law student who tabled the motion, said graduates felt neglected and Khan should choose between his political career and his position at the university.
"He has continuously missed graduation ceremonies which is his utmost duty as chancellor," Tanveer said.
"Nobody is against him. Students are unhappy with him for not fulfilling his duties, its best for both parties that he stand down," he said.
More From This Section
"Our chancellor has a major political role in Pakistan, a country which has serious problems with terrorism, education and poverty," Cantor said in a statement.
"While I understand that it is frustrating for students that Imran has not attended an award ceremony at Bradford for some time, I have urged students to be sympathetic to the situation in Pakistan and the responsibility Imran has to improving the state of the country."
"I have appealed to the better nature of our student body to be more supportive of the work Imran has to carry out as a politician," he said.
His predecessors include former British Prime Minister Harold Wilson, who rarely missed a ceremony.