The 47-year-old hip-hop icon is one of the inductees of the year 2017 to become the part of the prestigious line-up, reported Billboard.
The list was announced by Chic guitarist Nile Rodgers, who said it took longer for traditional voters to consider the 21-time Grammy winner fit for the honour.
He added Jay Z "was in a space where, even though he's had more pop albums than anyone else, because he did it through rap".
His best work includes tracks like "Empire State of Mind," which is a duet with Alicia Keys, "Off Your Shoulder" and "03 Bonnie & Clyde" with singer wife Beyonce.