The composer wrote a harmony for the Gregorian chant Pange Lingua and modified its tune, said the University of Manchester's Professor Barry Cooper.
The passage was written in a sketchbook but had previously been regarded as an exercise or a sketch for another piece, the BBC News reported.
Cooper recognised the hymn's tune and identified the two-minute piece as a previously unknown Beethoven work.
The leading expert on the composer said it was a rare example of the composer writing religious music.
"I was very surprised when I found this - I didn't think he'd written anything like this," he said.
The sketchbook, which is held at the Berlin State Library, dates from around 1820. It was transcribed and published in 2010.
More From This Section
Cooper said Beethoven scholars had not identified the piece earlier because "it just looked like a series of notes".
"When I looked at it, I thought, hey, that's not just a series of notes, that's a hymn tune with Beethoven's harmony," he said.
"It seems that people who are experts in plainsong don't look at Beethoven sketches very much and people who are experts in Beethoven sketches don't look at plainsong very much. But I happened to know both," he added.
Beethoven made slight changes to the tune of Pange Lingua, a 1,000-year-old Gregorian chant, Cooper said.
It may have been played at the ceremony where Beethoven's patron Archduke Rudolph of Austria was made Archbishop of Olmutz in March 1820, when the composer was 49, he added.
Experts in Berlin had assumed the piece was a sketch for his celebrated mass Missa Solemnis, which was written around the same time.
"He never composed any functional music at all - the only liturgical music he wrote was for the two great masses. So to find him writing simple functional music for an ordinary church service is extraordinary," he said.
Professor William Drabkin from the University of Southampton, who published the sketches, described it as a "very rudimentary piece".
"When I did the sketch transcriptions, I thought it was some sort of exercise in counterpoint. I couldn't place it," he was quoted as saying by the BBC.
"It's certainly a piece that we've not come across. It adds a small contribution to our knowledge of what Beethoven was doing in the 1820s," he said.
The hymn will be performed at the University of Manchester.