Malmö needed to beat the Greeks in order to stand any chance of finishing third in the group and landing a Europa League spot.
In a thrilling match the Swedish visitors equalised on two occasions with goals from Simon Kron and Markus Rosenberg making it 2-2 with just ten minutes remaining in the contest.
As Malmö pushed for an unlikely win, they were punished by Olympiakos who struck twice in the last three minutes. The 4-2 defeat means Malmö finish bottom of the group which was topped by Atletico Madrid and Juventus, with Olympiakos taking the Europa League place.
The team from southern Sweden was the first Swedish side to reach the Champions League group stage since Helsingborgs IF in 2000-2001, lost five of their six matches against the European big guns.
"It is never nice to finish last in the group, but we are a small team and got much experience from this season. I hope that we can be here next year as well," striker Marcus Rosenberg told Uefa.com.
As reigning Swedish champions, Malmö will get another chance to qualify for the Champions League next summer when qualifiers begin for the 2015/16 tournament.
Sweden's domestic Allsvenskan league championship starts again in March.