The 'Fashion Police' star was contacted by Stewart's camp, who threatened to sue her for defamation unless she removes the part from her book 'Diary of a Mad Diva', reported TMZ.
Rivers, 81, said the book is meant for laughs and it is stated in the prologue. It is not clear whether Stewart will go ahead with the case.
Rivers, known for her controversial comments, poked fun at Stewart's affair with married director Rupert Sanders who worked with her in 'Snow White and the Huntsman'.