Hagel told reporters flying with him to Poland that at some point Karzai's indecision will interfere with Washington's need to plan the post-2014 military mission that Karzai himself has said he favors.
"You can't just keep deferring and deferring, because at some point the realities of planning and budgeting, it collides," Hagel said aboard his plane.
He said US officials, including Gen Joseph Dunford, the top US military commander in Afghanistan, have pressed Karzai and "talk with him constantly."
He added that US allies who are willing to help train and advise Afghan forces beyond 2014 also are eager to know if there will be a US-Afghan security agreement soon.
The US now has about 39,000 troops in Afghanistan but would reduce that figure to zero by year's end unless a security accord is signed in the months ahead.
Hagel was visiting Warsaw to consult with Polish officials on Afghanistan and other security issues.