Fourth-tier Newport, one of several Welsh clubs who play in English domestic football, put themselves in sight of one of the all-time great upsets against eight-time FA Cup winners Tottenham thanks to Padraig Amond's first-half goal.
But eight minutes from time, in-form England striker Kane equalised when, after Spurs had won a corner, he tapped in from close range following an excellent flick-on from South Korean Son Heung-Min.
And 1-1 was how it finished as League Two Newport survived four minutes of injury-time to hold out for a lucrative replay at Wembley, Spurs' temporary home while their new home ground in north London that will replace White Hart Lane is built.
Ben Tozer's long throw into the box caused consternation in Spurs' defence and Robbie Willmott's clip across to the back post was nodded in by Amond.
More From This Section
It seemed Rodney Parade, where Newport Rugby Club beat giants New Zealand and South Africa in the 1960s, was about to play host to another great sporting upset.
But with County in sight of the last 16, boyhood Spurs fan Kane spared his side from a damaging defeat.
========================
Earlier, West Ham manager David Moyes labelled Arthur Masuaku "despicable" after the midfielder was sent off for spitting in a 2-0 defeat by Wigan.
Premier League West Ham, already 1-0 down to the first of two goals from Will Grigg, were reduced to 10 men four minutes into the second half when Masuaku saw red for spitting at third-tier Wigan's Nick Powell.
"It was despicable," said Moyes. "If you do that, then you're going to get the punishment in any walk of life.
"I just asked him, did he spit and he said, 'Yes'."
Wigan have slumped down English football's ladder since winning the FA Cup in 2013, but the League One leaders rekindled memories of that run to Wembley.
It took Northern Ireland striker Grigg just seven minutes to score his fifth goal in as many Cup ties this season with a header beyond England goalkeeper Joe Hart.
Grigg made it 2-0 from the penalty spot in the 62nd minute after Reece Burke's handball in the box.
"From start to finish we were the best side," Grigg told the BBC.
Elsewhere, struggling Premier League sides Swansea and Huddersfield played out 1-1 draws with lower-division Notts County and Birmingham City respectively, leading to unwanted replays for teams whose priority is remaining in the top flight.
Yesterday's early kick-off saw Fousseni Diabate score twice on his Leicester debut as the Premier League club won 5 -1 at Peterborough.
Nigeria's Kelechi Iheanacho also scored twice for the Foxes, with Wilfred Ndidi making it 5-1 in stoppage-time.
Nottingham Forest's FA Cup run ended just one round after they knocked out holders Arsenal courtesy of a 2-1 defeat by Championship rivals Hull.
Southampton beat Watford 1-0 as Jack Stephens marked his 24th birthday with a goal to ensure Javi Gracia's reign as Hornets manager started with a defeat.
Chelsea face Newcastle in another all top-flight clash on today when league leaders Manchester City continue their bid for an unprecedented 'quadruple' away to second-tier Cardiff.
Manchester United made it through to the fourth round on Friday, with new signing Alexis Sanchez impressing on debut in a 4-0 win away to lowly Yeovil Town.