The referee was arrested by police on Thursday night and detained for questioning on suspicions of sexual molestation.
He was released on Friday, however, it remains unclear why the official pulled his pants down in front of hotel cleaners.
Staff members at the hotel in Gothenburg where most of the World Championship teams and officials are staying complained that the referee had exposed himself in front of them.
The cleaners relayed their complaint to hotel managers, who in turn called the police, according to TV4 news website Nyhetskanalen.se.
The referee's behaviour was unwelcome news for the International Handball Federation (IHF), which decided to immediately suspend this referee for the entire World Championships, according to a statement on its website.
According to Nyhetskanalen.se, the referee traveled to Copenhagen on Friday in order to catch a flight to his home country.
The IHF has also asked for a report from the World Championship arrangers to determine what further action may be taken against the referee.
After tying for the top spot in its preliminary group with Poland, Sweden currently finds itself tied for second place in main round competition with Poland and Argentina behind Group II leaders Denmark.
The tournament, which began on January 13th, concludes on January 30th with gold and bronze medal matches to be played in Malmö in southern Sweden.