The youth, who is between 15 and 18, also allegedly suffered an assault and threats of further violence while staying at the centre in Årjäng, Värmland, close to Sweden’s border with Norway.
The men were seized on Sunday evening and detained in custody after questioning.
According to local police, the men, all of whom were asylum seekers at the home, were originally arrested for attempted rape.
Police later downgraded the charges to aggravated sexual assault, aggravated assault, and threatening behaviour following interviews with the five men and witnesses.
“It was not the kind of violence which is required for a rape,” Tommy Lindh from the Värmland police explained to The Local. “There's a difference between rape and aggravated sexual assault. You can make a person commit sexual acts without using violence, for instance.”
He said that police were not yet ready to give any information on the identities of the arrested men or to reveal which of the asylum centres in Årjäng had been the site of the crime.
Interviews with the five suspects, the victim and witnesses were continuing on Monday.
Police in Sweden are complaining of a sharp increase in their workload since more than 130,000 of asylum seekers arrived in the country last year, a number unprecedented in the country's history, with a string of crimes committed at asylum centres across the country.
Lindh said that local Värmland, where large numbers of asylum seekers have been housed, was no different.
“Of course it's extra work for the police. There's an increased population and the kind of living that makes people commit crime, so it's much harder work for the police,” he said.
“The police haven't been given any more resources or money locally: we have to deal with it with the resources we already have.”