A man in his forties was pushed to the ground by a supporter of the group and five people are suspected of committing assault, reports local media Oskarshamns-Tidningen.
Physical confrontation broke out when the man asked the neo-Nazi group to stop filming him, according to the report. When the group refused, he tried to take the camera from them and was then attacked.
Police have seized a camera containing footage documenting the disturbance.
“It has been confiscated as evidence in case there is film or pictures of what happened,” police spokesperson Stephan Söderholm told Oskarshamns-Tidningen.
The man who was attacked did not suffer serious injury.
Also on Saturday, a man wearing a t-shirt that read "Revolution: Support the Nordic Resistance Movement" walked on to the court during the Swedish Open tennis match between David Ferrer and Fernando Verdasco, yelled Nazi slogans and raised his arm in a Nazi-like salute.
Security guards and police officers escorted the man away as incredulous commentators decried the incident as an "absolute scandal".