The extremist group could plant explosives on trains and tunnels or sabotage tracks and electrical cabling, said Bild, Europe's most widely read daily.
While Germany said its threat level had not changed and Austria said no additional security measures had been taken, a Czech rail official said authorities there had implemented new, unspecified security measures.
Bild said the information came from the National Security Agency (NSA) in the United States, which had listened in to a conference call involving top Al-Qaeda operatives.
The attacks on Europe's rail network were a "central topic" of this call, Bild said.
More From This Section
According to US media reports, intelligence services intercepted a conference call earlier this month between Al-Qaeda leader Ayman al-Zawahiri and more than 20 operatives from across the group's global network.
This prompted the US and several allies to shut embassies across the Muslim world, fearful of a major attack.
On August 1, the US issued a worldwide alert, warning of Al-Qaeda plans to launch an attack in the Middle East or North Africa.
The ministry said that while security warnings were received from time to time, it would not comment on these, adding that generally speaking Germany was among the potential targets of international jihadist organisations.
The Austrian interior ministry meanwhile said there were "no grounds for concern". While authorities were in contact with their counterparts in neighbouring countries, no special security measures had been taken for trains or stations.
The Czech railway however said it had beefed up security after the Bild report.
"We took some preventive measures to not be caught by surprise," said Jakub Ptacinsky, spokesman for the SZDC company in charge of Czech railway infrastructure.