Reinfelt started his last appearance with a ten-minute speech listing how Sweden has changed under his leadership.
He said the country had become more gay-friendly and more open to foreign cultures both abroad and at home. He was given a rapturous applause by Parliament.
Fredrik Reinfeldt will formally leave his role as Moderate Party leader in March but he has already made it clear that this will be the last time he will take to the stand in Parliament.
Löfven says farewell to Reinfeldt. Photo: TT
Sweden's new Prime Minister has no previous parliamentary experience as this is the first time he has been elected as a Member of Parliament. But he has previously taken on Reinfeldt in numerous radio and television debates.
In his debut debate on Wednesday, Stefan Löfven said:
"The last government had good ambitions, but the result, from our perspective, was weak".
He said that the Social Democrats would take a different approach when tackling Sweden's growing unemployment and tumbling school results.
"Fredrik Reinfeldt has been the Prime Minister for eight years, he has my utmost respect, as a person, an ideologist, and as a representative for Sweden abroad," he said.
Löfven then added that he had heard Reinfeldt was a big fan of music, and gave him tickets to a new musical by Swedish comedian Jonas Gardell.
Centre Party leader Annie Lööf presented Reinfeldt with a casserole dish. Photo: TT
Later, Sweden Democrat leader Jimmie Åkesson gave Reinfeldt a framed certificate, which contained details of his donations to a range of humanitarian aid efforts to Syria.
Green Party co-spokesperson Åsa Romson thanked Reinfeldt and offered him coffee-table book about Swedish nature.
There were strong words of praise from the leaders of the centre-right Alliance parties that were in coalition with Fredrik Reinfeldt's Moderate Party until September.
Centre Party leader Annie Lööf, said: "You're a wonderful person Fredrik, and I'll never forget how much time you made for your children during our campaigning in the Alliance."
She gave him a casserole dish and some traditional Swedish foods, including a falukorv sausage.
"I hope you can make some dinner masterpieces with this in the future," she added, to giggles from Parliament.
Liberal Party leader Jan Björklund, gave him a coffee maker and a shirt that said "World's Best Friend" in Swedish.
"This is how we truthfully see you in the Alliance," he said.
Christian Democrat leader Göran Hägglund presenting a tool belt. Photo: TT
Meanwhile Christian Democrat leader Göran Hägglund talked about how the Alliance had worked to build a better Sweden. He then presented the outgoing Prime Minister with a tool belt, adding: "I hope you can continue with the building, wherever you end up working next."
Jonas Sjöstedt, the Left Party's leader, said he was most impressed by the way Fredrik Reinfeldt dealt with the rise of the nationalist Sweden Democrat Party.
"You chose never to back down to the racists. Some parties around the world have backed down, and it's been bad for all of them. The ball has started rolling and it's going downhill. You have acted honorably, and I hope the Moderates continue in the same direction," he said.
For his present, he offered a political book called The Reinfeldt Effect, which was written by another Left Party member.
"I hope you have the time to read this," he said as Members of Parliament laughed again.
After a light introduction to the debate, matters took a turn as Prime Minister Löfven tackled plans to stop Isis, how Sweden will help fight the outbreak of Ebola in West Africa, and promises of better welfare and education in Sweden.
In improving Sweden, Löfven referred to the Social Democrats' plan of attack as "The Swedish Promise" (svenska löftet), which he compared to The American Dream.
Alliance party heads and the leader of the Sweden Democrats were unimpressed. The Moderates have decided to abstain from comment in the debate.