An east London estate, where 700 people live, has received leaflets saying a "Higher Velocity Missile system" could be placed on a water tower, the BBC reported.
A spokesman said the Ministry of Defence had not yet decided whether to deploy ground based air defence systems during the July 27 to August 12 London Olympics.
A resident Brian Whelan said he has seen soldiers carrying a crate into the building. He also said his property management company put up posters and gave out the leaflets yesterday.
The leaflet states that the Armed Forces will be at the location for a military exercise between May 2 and May 7.
It goes on to say there will be a "major national exercise" from May 2 to May 10 to test the Armed Forces' capabilities for providing security during the Olympics.
The document added that if the government decides to use the missiles during the Games, the soldiers could be "operationally deployed for a period of up to two months this summer".
More From This Section
The weapon being considered is a High Velocity Missile (HVM) system, which would be based on the Lexington Building Water Tower. The tower contains residential flats.
The MoD says that the missiles will not pose a hazard to residents and "will only be authorised for active use following specific orders from the highest levels of government in response to a confirmed and extreme security threat".
The document states: "Having a 24/7 Armed Forces and police presence will improve your local security and will not make you a target for terrorists.
"The location has been chosen as it is situated close to the Olympic Park and offers an excellent view of the surrounding area and the entire sky above the Olympic Park.
"The top of the tower also offers a flat, uncluttered and safe area from which to operate."
The Army website says the HVM system is "designed to counter threats from very high performance, low-flying aircraft".
A MoD spokesman said: "As announced before Christmas, ground based air defence systems could be deployed as part of a multi-layered air security plan for the Olympics, including fast jets and helicopters, which will protect the skies over London during the Games.