Favreau directed the first two installments of the Marvel comics movie adaptation, but announced in 2010 that he would not be taking charge of "Iron Man 3" so he could work on other projects.
Shane Black was brought in as his replacement and production began in North Carolina, The Hollywood Reporter said.
However, Favreau hasn't cut all his ties to the upcoming sequel, he has been invited back to portray Happy Hogan once more.
The director, who will also serve as executive producer on the new film, made a cameo as the character in the 2008 original, and enjoyed more screen time as Hogan in "Iron Man 2".
Actors Guy Pearce and Ben Kingsley have also been added to the new cast of Iron Man 3, which will see Downey Jr return as the titular superhero, with Gwyneth Paltrow as his love interest and assistant, Pepper Potts.
The film is due for release in May, 2013.