David Farnell, 56, a convicted child sex offender and the biological father of the boy, Gammy, told Channel Nine he and his wife Wendy had "wanted to bring him with us".
It was their first interview since sparking global controversy after leaving the boy in Thailand with his 21-year-old Thai surrogate mother Pattaramon Chanbua and taking only his healthy twin sister Pipah back home.
"We never said you (Pattaramon) can have this baby, no matter what," said an emotional David Farnell.
The Farnells, from Bunbury south of Perth, Western Australia, previously claimed they were told Gammy had a congenital heart condition but not Down's syndrome, and left him because doctors said he would not survive.
More From This Section
Gammy, now seven months old, has since been cleared of a heart condition by Bangkok specialists, an Australian charity which has raised more than US$240,000 for the infant said on Friday.
David Farnell claimed Pattharamon said: "If we try to take our little boy, she's going to get the police and she's going to come and take our little girl ... And she's going to keep both babies."
Pattaramon has said she agreed to carry another Thai donor's egg fertilised by the Australian man in exchange for around USD 14,900.
An agency, which she refuses to name for legal reasons, acted as the go-between.