The court imposed an 800,000-ruble fine (about USD 14,000) over Telegram's failure to "provide law enforcement agencies with information" about its users and their messages, TASS news agency reported.
The free instant messaging app, which lets people exchange messages, photos and videos in groups of up to 5,000 people, has attracted about 100 million users since its launch in 2013.
Telegram's self-exiled Russian founder Pavel Durov said in September the FSB had demanded backdoor access.
According to a scan of the complaint posted online by Durov, the FSB had sent a letter to Telegram in July demanding "information necessary to decode users' sent, received, delivered and processed electronic messages".
More From This Section
The fine is the latest move in an ongoing dispute between Telegram and the Russian authorities as Moscow pushes to increase surveillance of internet activities.
In June, Russia's state communications watchdog threatened to ban the app for failing to provide registration documents. Although Telegram later registered, it stopped short of agreeing to its data storage demands.
Starting from 2018, they must also keep all data from users in Russia according to controversial anti-terror legislation passed last year which was decried by internet companies and the opposition.
Telegram now has 10 days to appeal today's decision. If an appeal fails, the company will be given a grace period to hand over its encryption keys after which it could be blocked in Russia.
Asked about a potential block of the service, Kremlin spokesman Dmitry Peskov said: "As far as I know... there is no discussion of a block at this time.