The barrier, due to be completed in less than six weeks, will be the first fence between two members of the passport-free zone.
Austrian Interior Minister Johanna Mikl-Leitner insisted the "fence conforms to the Schengen accord", adding that it was part of temporary measures aimed at "channelling" the human flow.
"We are talking here about an ordered inflow and not a barrier," Austrian Chancellor Werner Faymann's chief of staff, Josef Ostermayer, told reporters in Vienna.
Austria's decision is the latest in a series of tough measures taken by countries to tackle the continent's worst migration crisis since World War II.
More From This Section
Earlier this week, Sweden - a preferred destination for migrants - reinstated temporary border controls, while Slovenia rolled out razor wire along its frontier with non-Schengen member Croatia.
Fellow bloc member Hungary already sealed its southern border with razor wire last month, diverting the influx toward Slovenia.
Austria's mesh fence either side of the Spielfeld border crossing point in southern Austria will be 2.2 metres high.
Barbed wire will be stored in nearby containers ready to be rolled out along the border - within the passport-free Schengen zone - if the situation escalates, officials said today.
Austria had initially planned to install a 25-kilometre fence but "our Slovenian colleagues have asked us to not do this immediately," said Austrian Interior Minister Johanna Mikl-Leitner.
However Austrian authorities were ready to unfurl the entire 25-kilometre fence if Slovenia's measures failed to control the flow, she stressed.