The Australian Broadcasting Corporation cited people close to the families of Khaled Sharrouf and Mohamed Elomar as saying the pair died in fighting in the Iraqi city of Mosul in the past week.
Foreign Minister Julie Bishop confirmed there had been recent drone strikes in the area but said she was still awaiting "absolute verification" they were dead.
"The likelihood of verification in relation to Mr Elomar is probably imminent, however in relation to Mr Sharrouf, we're still seeking to verify the reports," she said.
Dressed like any other youngster in blue checked trousers, a blue shirt and baseball cap, the picture of the child was captioned "That's my boy", prompting US Secretary of State John Kerry to call it "stomach-churning".
Also Read
Other photos, thought to have been from Syria, published in Australia at the time showed Sharrouf dressed in camouflage fatigues posing with three young boys, all believed by security agencies to be his sons
Elomar, who reacted to the photo of the boy with a tweet to "keep them heads rolling", has also been pictured holding heads of pro-Syrian fighters.
"Both Mr Elomar and Mr Sharrouf rose to international infamy as a result of pictures that they posted of themselves holding up the heads of pro-Syrian fighters. They had been notorious for their violence and their barbarity," said Bishop.
"These two men are not martyrs. They are criminal thugs who have been carrying out brutal terrorist attacks, putting people's lives in danger."
"It's a good day isn't it," added Peter Nettleton, whose estranged daughter Tara is married to Sharrouf.