While the new mission would have a military component it would notably be led by a civilian, Stoltenberg said after a session of the NATO foreign ministers meeting in the Turkish city of Antalya attended by Afghan Foreign Minister Salahuddin Rabbani.
NATO is currently leading the Resolute Support mission in Afghanistan to train Afghan security forces after the International Security Assistance Force (ISAF) wound up late last year.
"We took a major decision," Stoltenberg told reporters. "We will maintain our presence in Afghanistan even after the end of our current mission."
He said the future mission "will have a light footprint but will have military component."
More From This Section
Stoltenberg said NATO offices had been tasked with working out a plan of the new mission by the autumn. It is not yet immediately clear when the Resolute Support mission will end and the new mission could start.
"The aim will be to advise Afghan institutions to help them become self-sufficient and help them build on what they have done so far," he said.
He said NATO and the Afghan government had succeeded today to agree to the "guidelines" of the new mission.
With deadly Taliban attacks on the security forces a regular occurence, Stoltenberg acknowledged that Afghanistan remained a "dangerous place".
He said the Afghan security forces hade made "enormous sacrifices" and NATO allies and partners "will stay the course."
ISAF was established by UN Security Council resolution in 2001 in the wake of the US invasion of the country to provide security and fight the Taliban.