Erkan Zengin broke a nil-nil deadlock after 34 minutes with his second goal in three qualifying games.
Jimmy Durmaz scored the other goal in the first minute of the second half at the Friends Arena in Solna in the Swedish capital.
Goalkeeper Andreas Isaksson captained the side, with star striker Zlatan Ibrahimovic absent due to injury.
Sweden now have five points and are third in Group G.
Elsewhere in the group, Austria went top with seven points from three games after edging Montenegro 1-0 in Vienna thanks to a first-half goal from Rubin Okotie, who converted a 24th-minute pass from Marko Arnutovic.
Montenegro star striker Mirko Vucinic saw two good chances well-saved by Robert Almer in the Austria goal.
Previous leaders Russia were held to a second successive 1-1 draw and have five points from three games.
Sweden's Zlatan Ibrahimovic, who plays for Paris Saint-Germain has missed his club's last four matches, including the 3-2 Champions League win over Barcelona, and Sweden's 1-1 draw with Russia on Thursday.
"Unfortunately, we can't count on Zlatan Ibrahimovic. His injury is stopping him from playing but he will be on the bench with the rest of the team," the Swedish football federation said ahead of the match.
Sweden kicked off their Euro 2016 qualifying Group G with a 1-1 draw in Austria last month before Thursday's draw with Russia.