Interior Ministry spokesman Brig. Gen. Saad Maan said the grave in central Ramadi contained the bodies of police and civilians, including women and children.
Video footage from the site shows local security forces and a small forensics team wearing face masks and digging with shovels.
Ramadi, the provincial capital of the sprawling Anbar province, fell to IS in May, marking a major setback for US-backed Iraqi forces. Iraqi troops retook the city center last month with heavy coalition air support.