The Malmö native struck with just four minutes remaining to net the Swedes three vital points at the Friends Arena. Austria had taken the lead in the first half through Martin Harnik and led at the break.
Sweden equalized early in the second half when Martin Olsson got on the end of an Ibrahimovic cross to score with a diving header.
The hosts began to take control of the game and would have taken the lead but for a series of outstanding saves from Austrian goalkeeper Robert Almer.
With the match looking like it was going to end in a draw, leaving both sides with 17 points, the stage was set for Ibrahimovic.
The Paris Saint Germain striker latched onto a long pass from Kim Källström and finished cooly from close range to send the home fans into ecstasy.
The Swedes will wrap up their campaign on Tuesday against group winners Germany at the Friends Arena in Solna.
German coach Joachim Löw told AFP that his side are eager to make amends for the 4-4 draw in Berlin last October when Sweden fought back from 4-0 down.
“The (German) team has done well and we’re happy to qualify, but we definitely have a score to settle against Sweden,” said Löw.
“I’ll see how the players come out of this match, I might switch things up depending on that, but we want to win in Sweden,” he said.
Sweden take on Germany at 20:45 on Tuesday night.