The study from the SOM institute at Gothenburg University found that while 49 percent of Swedes were concerned about immigration levels, 78 percent were worried about an increase in racism.
The negative attitude towards xenophobia is likely due to the fact that the topic has been a hot one for the past two or three years, Marie Demker, political scientist, told Sveriges Radio (SR).
"I noticed that people feel that xenophobia threatens society," she told SR.
"We talk an awful lot about xenophobia and there is also a strongly negative attitude to all forms of racism and xenophobia. I think you can put it all down to the fact that xenophobia has been so high on the agenda over the past few years."
She added that it was "quite clear" that Swedes were more worried about other people's attitudes towards immigrants and refugees than they were about the foreigners themselves.
Immigration to Sweden has been a hot topic in Sweden, not least with Sweden taking the lead in questions of granting asylum to refugees. Last week, Eurostat revealed that Sweden took in 20 percent of the EU's asylum seekers in 2013 - a total of 26,395 asylum seekers, mostly from Syria. Around 15 percent of Sweden's population is foreign born.
The study, which marks the first time Swedes' concerns about xenophobia have been measured, will be published in full later on Tuesday.