The 17-time Grand Slam champion, seeded 17, fought back from a break down in the final set to win 7-5, 6-3, 7-6 (7/3) and will next face long-time rival Tomas Berdych.
Federer, 35, had his service broken for the first time in the final set and the 200th-ranked Rubin was serving to take it into a fourth set before the Swiss legend clawed back to force a tiebreaker.
His match-big experience and composure was enough to pull him through and take the match in 2hr 4min on Rod Laver Arena.
"It's great to get to see a few matches now. Yeah, plays well. Great fighter. Great legs. Aggressive on the ball. My serving kept me in the match today and happy I got through somehow."
It was Federer's second win against a qualifier at this year's Open after he defeated experienced Austrian Jurgen Melzer in four sets on Monday.
Disclaimer: No Business Standard Journalist was involved in creation of this content