The former welder got 132 yes votes. 49 people voted against him and 154 abstained.
Fourteen of Parliament's 349 MPs were not present for the vote.
The abstentions came from the Alliance parties that made up the previous centre-right government, the Moderates, the Christian Democrats, the Liberals and the Centre party.
"Wow, how truly great and momentous this is," Stefan Löfven told reporters after his appointment was agreed on.
All members of nationalist Sweden Democrat Party chose not to vote for the Social Democrat leader.
Sweden's new Prime Minister with parliament's Speaker. Photo: TT