"Some groups such as youth with limited education, some immigrants, and those on sickness and disability benefits are not well integrated," the Organization for Economic Cooperation and Development said in the report.
To boost employment in those groups it said Sweden should lower wages for entry-level jobs, improve vocational and academic training, and ensure active labour market policies were better targeted to individual needs.
Sweden, famous for its egalitarian policies, also needed to reduce the gap in job protection between temporary and permanent employment so that people were not trapped in temporary contracts, it said.
The Paris-based organization added that "relatively high minimum wages set through collective bargaining tend to push up labour costs", noting that minimum wages in Sweden averaged around two thirds of the median salary.
Although the country's wage setting has become more decentralized, that ratio was still the second highest in the OECD in 2010.
"The government should continue talks with social partners to find ways to improve job prospects for groups at risk," it said.
The OECD also warned that Sweden's large banking system "entails risks and potential costs", singling out high household debt as a risk should housing prices fall or unemployment rise.
The Swedish unemployment rate stood at 7.6 percent last year, compared with an 8.2 percent average for the organization's member countries.
However, youth unemployment was comparatively high at 22.9 percent, compared with 16.2 percent in the OECD.
The OECD also said it expects Swedish economic growth to drop to 1.2 percent this year, down from a growth rate of 3.9 percent in 2011.
Growth is expected to recover somewhat in 2013, rising to 1.9 percent before increasing to 3.0 percent in 2014, according to the OECD.