A 49-year-old father, his two sons, aged 27 and 21, as well as his 33-year-old son-in-law, were arrested on Monday on suspicions of extortion, the Aftonbladet newspaper reported.
The men allegedly prowled the internet pretending to be children and then threatened to reveal the identities of men who had sought sexual contact with them.
"Some have paid a lot of money not to be unmasked as paedophiles," one source told the newspaper.
Police learned about the scheme, alleged to have started back in August 2009, through a tip from one of the blackmail victims.
But acquaintances had also begun to suspect the modest, working-class family was engaged in some sort of illicit activity after the family, which had previously lamented out their dire financial straits, went on a spending spree over the last two years, purchasing cars and going on trips.
"This is a normal worker, a family man. No one could believe they'd go so far. But of course people noticed that they suddenly seemed to have plenty of money," a family friend told Aftonbladet.
The 21-year-old son, who has confessed to the blackmail accusations, campaigned for a seat in the Riksdag as a member of the neo-Nazi Svenskarnas parti ('The Party of the Swedes').
The 49-year-old father also admitted to the scheme during a remand hearing on Thursday, while his other son and son-in-law, denied committing any crime, the local Norrköpings Tidningar newspaper reported.
Prosecutors expect to formally charge the men within the next two weeks.