The four-time champions dominated possession but struggled to create enough clear-cut chances, as Sweden secured a first appearance at the finals since 2006 courtesy of Jakob Johansson's first-leg strike in Stockholm.
Despite the support of 74,000 fans in a white-hot atmosphere at the San Siro the 2006 World Cup winners could not find a way through.
It is the third time that the Italians will not compete at the World Cup, after not going to the first edition in 1930 and not qualifying for the 1958 finals in Sweden.
At 39, the 175-times capped Buffon had announced that the finals in Russia would be his last as a player for the national side.
More From This Section
"I'm sorry, sorry, sorry," a tearful Buffon told Rai television as he confirmed he was ending his international career.
"I'm not sorry for myself but all of Italian football, because we failed at something which also means something on a social level.
"It's the only regret I have. Not stopping, because time passes and it's normal. I regret just that my final match with the national side coincides with an elimination.
Without star Zlatan Ibrahimovic, who retired from the national team after Euro 2016, Sweden sealed a return to the world stage after failing to reach the last two editions. The last time the Scandinavians participated was in 2006.
"I'm emotional and very happy," said Sweden coach Janne Andersson. "This match showed our collective strength.
"With Ibrahimovic we played differently. He's retired and he's a great champion. So we adapted and we play in another way now."
Mikael Lustig almost scored an own goal when his deflection hit the bar.
Marco Verratti sat out a ban with Simone Zaza and Leonardo Spinazzola both injured, with Daniele De Rossi and Andrea Belotti not fully fit.
Coach Gian Piero Ventura gave Brazil-born Jorginho his first competitive appearance for Italy with Florenzi and Manolo Gabbiadini also handed starts. Napoli's in-form winger Lorenzo Insigne however was once again left on the bench.
Sweden's Lustig was back from suspension, with Johansson starting in place of the injured Albin Ekdal.
Sweden lost first-leg goalscorer Johansson after 15 minutes when his left knee buckled badly and he was stretchered off to make way for Gustav Svensson.
Jorginho's through ball found Immobile to pull back from the byline for Candreva to blast over, but Leonardo Bonucci, wearing a protective mask after breaking his nose on Friday, got a knock to the knee and was limping badly.
Florenzi thought he had scored on 53 minutes with a flying volley from Matteo Darmian's cross, but it went wide.
Federico Bernardeschi was the final Italy substitution and Olsen turned away a Parolo header.
As the clock ticked the crowd at the San Siro began to sing the national anthem.
An El Shaarawy shot was kept out by Olsen and Parolo's header went wide, then Buffon came up for a corner in a final desperate gamble as Italy fell short of reaching the World Cup for the first time in 60 years.