Catharina Strandqvist, 20, who is also a soldier in the Swedish army, was standing in local elections in Halmstad in south west Sweden, but quit after pictures emerged of her wearing a swastika armband as she cleaned up after a party in 2012.
She has now stopped working as a home help assistant in Halmstad in southern Sweden after pensioners in the Karl XI service building complained about her. It is unclear if or when she will return to the post.
"She was really nice and good at her job, but when I saw the picture where she stood and hoovered, wearing an armband with the swastika, I felt immediately that I did not want her in my home," resident Nina Söderman, told local newspaper Hallandsposten.
"Hitler symbols are not something you can sweep under the carpet," added Söderman, who is a wheelchair user.
"I also think it is insulting to all her colleagues who have an immigrant background."
Another resident told the newspaper:
"I was a teenager when Hitler's troops marched into my country, so this raises dark memories for me".
Catherina Strandqvist has not responded to media interview requests.