Swiss coach and former international midfielder Marcel Koller guided the Austrians to a place in France after their maiden appearance at the final stages in 2008 when they qualified as co-hosts with Switzerland.
Bayern Munich's David Alaba, the youngest player to ever play for Austria at 17 in 2009, scored from the penalty spot after just nine minutes before West German-born Martin Harnik plundered his 12th international goal seven minutes before half-time with a header to give the visitors a cushion.
Veteran Basel striker Marc Janko, 34, then put the icing on the cake and started the early celebrations with Austria's third on 79 minutes before Harnik grabbed his second of the night with two minutes remaining to add insult to injury for the Swedes.
Swedish captain Zlatan Ibrahimovic scored a late consolation goal in vain for the hosts.
The 33-year-old Paris Saint-Germain striker came off at half-time in Sweden's Saturday's 1-0 defeat to Russia in Moscow, complaining of an apparent abdominal injury and was only confirmed as a player for the national side on Tuesday morning.
Swedish coach Erik Hamren opted to revert to a 4-4-2 formation after unsuccessfully trying out 4-5-1 in the Russia defeat.
On Tuesday Russia dispatched Liechtenstein 7-0 to move into the second qualifying spot, twopoints in front of the Scandinavians, in Group G.
In the pool's third match Montenegro won 2-0 on the road at Moldova as they climbed within a point of Sweden and a potential place in the play-offs with two matches to play.