While it is normal for batsmen to walk away from a delivery if anyone in the crowd interferes with their line of sight, Johnson said he suspects the regularity with which it has happened during the series indicates England are doing it for other reasons.
The fiery left-armer said he expects Alastair Cook's team to continue to do so in the fifth and final Test in Sydney starting on Friday. Australia are chasing a 5-0 Ashes series whitewash for only the third time against England.
"It definitely is frustrating when it happens all the time but that's part of the game, it's part of their tactics."
The issue came to a head during the fourth Test in Melbourne won by Australia by eight wickets, when star English batsman Kevin Pietersen walked away during Johnson's run-up.
More From This Section
It infuriated Johnson, who is the leading wicket-taker in the series, and he threw the ball in Pietersen's direction and exchanged words with the batsman.
"The only thing I regret is throwing the ball," he said.
"I think that was probably a little bit inappropriate but the rest of it was fine. I just let (Pietersen) know that he needed to stop doing it.
"The sight screens are big enough, he should be watching the game. I won't back down if it happens again."
Johnson, who has taken 31 wickets at 14.32 in the series, said he was spurred on by Pietersen's hasty retreat in Melbourne -- where he claimed three for 25 in the second innings.
Johnson needs nine wickets to become only the fourth Australian bowler to take 40 wickets or more in an Ashes series along with Terry Alderman (twice), Rodney Hogg and Shane Warne. Of those, only Warne took his 40 wickets in a five-Test series, a feat that Johnson is seeking to emulate.