In 2016 around 3,900 work permits were granted in Sweden for jobs which required no more than a short induction or period of training. A large proportion was in low-skilled agricultural work like berry-picking.
Backed by LO, the Social Democrat leadership now plans on proposing restrictions in the area at its next party conference.
"There are unemployed people in our country who could take these jobs, which don't require higher education or high levels of qualification. It is reasonable that they should get these jobs first," Social Democrat PM Stefan Löfven told Swedish news agency TT.
Sweden's current rules for migrant labour were put in place by the previous centre-right Alliance government, and were backed at the time by the Greens, who are now in coalition government with the Social Democrats.
That didn't help the Green Party's popularity with the blue collar LO unions, but the party does not plan on changing its stance on the matter, regardless of what its Social Democrat partners think.
"No, we have the same position as before. We think that today's rules are valuable and we are of course working in different ways to improve them, prevent people being exploited and improve processing times," Green spokesperson Maria Ferm told news agency TT.
PM Löfven insists his party is not trying to appeal to the voters lost to populists with the proposal, and argues that those who are unemployed in Sweden should be prepared to take the available jobs regardless of whether they prefer them or not.
"If you're unemployed in Sweden you must also be ready to take the jobs that there are, that's the rules we have," he said.
The centre-right opposition Moderates have questioned how effective limiting foreign workers would be in that regard however.
"The problem isn't that too many people are coming here and picking berries. The big problem is that far too few people who live here are taking the jobs that there are, like for example, picking berries," Moderate labour policy spokesperson Elisabeth Svantesson said.
Social Democrat members will decide whether to back the proposal or not at the party's next conference in Gothenburg this April.