Now hailed as a 80s classic and one of the highlights of the New Romantic era, the single was kept off the top spot by novelty track 'Shaddap You Face' by the US-born Australian Joe Dolce.
But the song, released in January 1981, has now triumphed in a national poll by Radio 2 and the Official Charts Company to find the greatest track to narrowly miss out on the top slot, reported Daily Mail.
"We are extremely pleased and very humbled to have been given this honorary number one, especially knowing the outstanding records which were also in the running - Strawberry Fields, Hound Dog and Wonderwall to name just a few," Ultravox frontman Midge Ure said.
"A huge thanks to everyone who took the time and effort to vote for 'Vienna'. It has restored our faith in the idea that the public will always recognise the good in music, and that this song has proved this," he added.
The Ultravox song's triumph left The Pogues and Kirsty MacColl single 'Fairytale Of New York' as the all-time number two number two.
The 1987 Christmas hit was originally held in check by the Pet Shop Boys cover of Elvis Presley's hit 'Always On My Mind'.
The poll was launched to celebrate 60 years of the singles chart.