Starc, who replaced Mitchell Johnson in the ongoing Sydney Test, had Vijay caught behind in the third ball of the Indian innings.
The left-arm paceman gave a hard look to Vijay as he shouted in zest with pumped up fists, celebrating the early success.
However, young captain Smith was not impressed with the act of his fast bowler.
"I don't think it's a good part of the game to send batsmen off. I didn't see what Starc did. I was busy celebrating myself. But it's not a part of the game that needs to go ahead. I'm sure our players will stay pretty calm when we get wickets from now on," Smith said at the press conference.
While the Indian bowlers were quiet throughout the first two days, decorum was broken by Starc.