Cristiane, the Olympics' all-time top scorer, and Andressa saw spot-kicks saved by Hedvig Lindahl after neither side could score a goal in 120 minutes.
Lisa Dahlkvist converted the winning penalty to secure a place in the final and Sweden's first women's footbally Olympic medal.
Germany face Canada later in Belo Horizonte in the other semi-final.
Brazil had thrashed the Swedes 5-1 in the group stages just ten days ago, but were held to a third consecutive 0-0 draw and, just like in their quarter-final win over Australia, forced to go through the agony of penalties.
Unlike against the Aussies, five-time World Player of the Year Marta did convert from the spot, but was reduced to tears as her ambition of finally winning a major international title fell short of the final.
Marta, who has played her club football in Sweden for the past four years, was consoled by the Swedish players, while a huge crowd in excess of 70,000 gave their defeated team a huge ovation.
Sweden coach Pia Sundhage now has the chance to win a third straight gold medal after leading the USA to victory in 2008 and 2012.
“It is very emotional. I have been in China and London in Olympic finals, but with a team everyone expected to do well. (…) The road we have taken has been different, a wonderful trip with one or two upsets,” she said.
Sweden's hero Lindahl described the baking sun in Salvador during the Scandinavians' shock quarter-final win over three-time defending gold medallists the United States as “her enemy”.
Lindahl suffers from vitiligo, a condition that causes patches of skin to lose their pigment, making the skin particularly sensitive in the sun.
And she was under fire in the Rio heat as a flurry of early Brazilian attacks peppered the Sweden goal.
Debinha fired a volley just over and forced Lindahl to tip over a header, while the Chelsea goalkeeper also nearly turned a dipping Marta corner into her own net.
Lindahl even had to endure chants of “Zika” from the home fans, which had been previously reserved for outspoken USA goalkeeper Hope Solo.
Marta had inspired the 5-1 thrashing of Sweden with two goals in the group stage, but despite their dominance the ball wouldn't go in for the hosts as Marta prodded wide and Beatriz – who also scored twice in the early win over Sweden – forced Lindahl into another routine save.
Solo had castigated Sweden as “cowards” for their defensive display in the last eight, but that seemed to only strengthen the resolve of Sundhage's women as they barely threatened the Brazilian goal.
Brazilian stopper Barbara – the hero fo an agonizing 7-6 shootout win over Australia – was only forced into action when her own poor clearance allowed Stina Blackstenius a shot on target which was easily mopped up.
Veteran midfielder Formiga, the only woman to play in all six Games since women's football was introduced to the Olympics, came close to a fairytale end when her low header was just kept out by Lindahl in the fourth minute of stoppage time.
Brazil coach Vadao introduced Cristiane for extra-time despite doubts over her fitness after missing the quarter-final through injury.
But it was Sweden who came closest in the extra 30 minutes when captain Lotta Schelin headed inches wide from a corner and then shot into the side-netting at the end of a sweeping counter-attack.
Marta saw a low free-kick held by Lindahl, but a rare flap by the keeper caused a flurry of blocked shots inside the Sweden area moments later as they held on for penalties.
Lindahl came up trumps in the shootout, though, and Dahlkvist's cool finish plunged a stunned Maracana into silence.