Ninth seed Nadal, a 14-time Grand Slam champion, defeated the sixth-seeded Monfils 6-3, 6-3, 4-6, 6-4 on Rod Laver Arena in almost three hours and will play Canada's world number three Milos Raonic in the quarters.
It was a gutsy effort from the never-say-die Spaniard, who was down a service break with a fifth set looming before he broke Monfils twice to clinch victory.
The 2009 champion reached the quarter-finals in Melbourne for the ninth time and it was his first since the 2015 French Open.
He added: "Gael is a special player. He is able to play unbelievable shots. It is a pleasure to see him in the top rankings."
More From This Section
It was Nadal's first win over a top-10 player at a Grand Slam since he defeated number two Novak Djokovic to win the title at Roland Garros in 2014, his last Grand Slam win.
He will go into his Australian quarter-final with a 6-2 lead over Raonic in their previous meetings.
Nadal lost his service in the sixth game but broke the unpredictable Frenchman three times to open up a two-set lead and was closing in on a straight-sets victory.
But Monfils fought back, breaking Nadal in the ninth game and saving three break points in serving it out to trail 2-1 in sets.
Nadal began to struggle in the fourth set as Monfils fought off two break points and broke the Spaniard in the next game with a net cord on one of his three break points.
Nadal then got to his first match point in Monfils' next service game, which was saved. On a second match point, the Frenchman's backhand was narrowly wide to send Nadal through.