Tablighi Jamaat's mosque with 190-foot minarets near the Olympic Park would have accommodated up to 9,300 worshippers in two main gender-segregated prayer halls and a further 2,000 in a separate hall.
The Department for Communities and Local Government (DCLG) has dismissed appeals against the refusal of planning permission for the mosque in Newham area of the city.
A DCLG spokesperson said: "The decision was based on concerns that include local housing provision and conflict with the council's local plan for the borough.
The project proposed by Tablighi Jamaat - referred to by a number of names such as Abbey Mills Markaz, London Markaz, Masjid-e-Ilyas or the Riverine Centre - was rejected by the local Newham Council back in December, 2012, with councillors saying the building was too large and would harm their plans for a mixed-use neighbourhood.
Also Read
Tablighi Jamaat had appealed, taking the application to a three-week public inquiry last year.
The inquiry inspector's report was submitted to the DCLG in January this year and the decision this week concludes a long-drawn battle over the project.
A temporary mosque known as the Riverine Centre, with a capacity of 2,500 people, was built on the site soon after.
The new mosquewas to have been its new headquarters with residential facilities, a library, visitor centre and sports centre, besides the mosque.
The conservative Tablighi Jamaat has been accused of links with Islamic radicalism, which is hotly disputed, with many experts saying it is peaceful and non-political.