"There are eight bodies which were shot execution style," Paul Ramaloko of an elite police unit told AFP.
The bodies were discovered by mine security between Saturday and Sunday.
Two of them were found underground and six on the surface at an abandoned gold mine in Benoni, east of Johannesburg.
Police spokesman Lungelo Dlamini said the eight had wounds on their upper bodies. They are yet to be identified.
"We don't know the motive behind the killings," said Dlamini.
Also Read
Fighting between competing gangs is commonplace among the illegal miners at South Africa's abandoned gold mines, where unemployed and impoverished people extract any remaining gold with makeshift tools.
Much of the area around Johannesburg is pockmarked with old shafts, a testament to the city's century-old history of gold mining.
While many are no longer commercially viable, they still contain enough deposits to attract scores of the city's unemployed, and those from much farther afield.
The owners of the mine could not be immediately verified.
The rescued men, many of them undocumented immigrants from neighbouring countries, were arrested and charged with illegal mining.