Bayliss, who was appointed in 2015 to succeed the sacked Peter Moores, said it would be time for a fresh approach when his four-year contract comes to an end.
"I'm contracted to September 2019 and that will see me out," the 55-year-old Australian told the BBC.
"I've always felt that it's time for a change around that four-year mark. A new voice and a different approach reinvigorates things."
The broadcaster said Bayliss told England director of cricket Andrew Strauss of his plans a year ago, but had only now made it public.
More From This Section
Since former Sri Lanka coach Bayliss took over from Moores, England's Test results have been mixed, losing 18 and winning 15.
His announcement follows England's chastening 4-0 Ashes defeat, with Australia recording four comprehensive wins -- including two innings victories and a 10-wicket triumph.
British media criticised Bayliss after Monday's loss in the fifth and final Test in Sydney, with the Daily Telegraph accusing him of being "laissez-faire".
Among England's Ashes low-points, vice-captain Ben Stokes was suspended before the tour pending an investigation into a nightclub brawl, and wicketkeeper Jonny Bairstow drew controversy when he greeted Australia's Cameron Bancroft with a headbutt in a Perth Bar.
"England have little chance of improving if they stick with Trevor Bayliss," said the Evening Standard, while the Independent asked: "What is the point of Trevor Bayliss?"
His decision will trigger speculation about a successor as England's management open their review of the failed Ashes series.
Former Australia fast bowler Jason Gillespie, who was appointed as interim coach of Papua New Guinea in July, was seen as a strong candidate when Bayliss got the job in 2015.
Bayliss' career with England began with a 3-2 home Ashes victory in 2015, but they suffered their first ever Test defeat to Bangladesh in 2016 and were humbled 4-0 away by India a year ago.