The 70-year-old's contract expires at the end of June and Cricket Australia directors will consider changes to the selection committee at a board meeting this week.
"The man most likely to step into the role would be Rod Marsh, who has served alongside him (Inverarity) since 2011, when the panel was revamped in the wake of the Argus review," Fairfax Media said.
Former Test batsman Mark Waugh could also be a significant addition as a selector should a proposal from CA team performance manager Pat Howard be ratified.
Part of Australia's golden era in the 1970s, Marsh played 96 Tests during a 14-year career and has a significant post-playing pedigree for the chairman's job.
Also Read
He headed up Australia's national academy for more than a decade, then took up the same position in England in 2001 before also becoming an England selector.
Marsh, 66, stepped down from the England academy and as a selector in 2005 after playing a key role in England's first Ashes victory for 16 years.