Moss says she did not defend herself publicly until now because she never cared about what people thought of her.
The 38-year-old fashionista says silent was the most effective weapon to deal with the rumours swirling around her alleged drug abuse, the Daily Mirror reported.
"If I was anorexic or if I was on heroin, maybe I would have been a bit more 'oh dear! But I wasn't any of those things that they were painting me to be," Moss says in her book.
"It didn't have anything to do with who I was at all, so I never really thought about it. I just thought, the people who know me know the truth."
Moss says that she never spoke out publicly to defend herself because she followed the advice of her former boyfriend Johnny Depp.
The Pirates Of The Caribbean actor had told her to "never complain" and "never explain".