Thiem looked headed out the door before he turned around the match in the third set and went on to complete a 6-7 (6/8), 3-6, 6-3, 6-2, 6-3 victory in 3hr 48min on Margaret Court Arena.
In searing on court temperatures, Thiem wore down the 190th-ranked Kudla to set up a third round match against Czech Jiri Vesely or French 26th seed Adrian Mannarino.
"It's awesome. It's only the second time in my career I've come back from two sets down and especially in this very hot weather it was tough," Thiem said.
"But it's difficult and we just have to hydrate and not try to play too long, which wasn't the way today. I'm off to have an ice bath."
More From This Section
Thiem reached the third round in Melbourne for the third straight year but had to pick up his service after firing eight double-faults.
He broke Kudla's serve eight times and lost his four times, but was very effective with his forehand, cranking up 21 of his 57 winners.
He arrived late in Melbourne from Doha after pulling out prior to the Qatar Open semi-final with Gael Monfils after contracting a virus.
Thiem was bed-ridden before flying to Australia, but said ahead of the year's opening Grand Slam he had recovered.