The 'Blind Side' star was in talks to portray mean orphanage boss Miss Hannigan alongside young actress Quvenzhane Wallis, but negotiations have fallen through, reported Daily Star.
It was Sony executives' second attempt to sign Bullock, 48, to the film as she also turned down the role in March.
They are now on the hunt for a suitable replacement to fill the role made famous by Carol Burnett in the 1982 film, and more recently portrayed on Broadway by 'Glee' star Jane Lynch.
Smith's actress daughter Willow had previously been cast as Annie but she pulled out of the project earlier this year.