The group was among 131 residents moved from the Klackbergsgården asylum home in Norberg, central Sweden, after a series of serious safety concerns were raised about living standards.
As The Local reported last week, the Migration Agency decided to close the centre and began moving people to other homes in Norberg and northern Sweden on Thursday.
But around ten people in a one group of 26, which included children, refused to get off the bus when it arrived at their new facility in Ytterhögdal in the Jämtland region, almost 300 kilometres north of Norberg.
Among their concerns they reported that there was only one bathroom for six families and that the closest food shop and hospital were located too far away from the centre.
Some later agreed to stay at the new centre, but four remained on the bus until the bus company called the police this weekend, reporting them for unauthorized use of the vehicle.
“The police came out and talked to the asylum seekers. After that everyone left the bus and the driver was able to leave,” Migration Agency official Guna Graufelds told Swedish radio.
According to police the remaining four people left calmly after being spoken to by officers.
The bus protest is not the first of its kind in Sweden. In October last year dozens of people refused to get off a bus that took them to a rural location in Sweden. The Migration Agency told The Local at the time that the refugees had said they felt uncomfortable being surrounded by the forest.