As part of a documentary, Swedish broadcaster TV4 filmed secret footage of the privately-run Al-Azhar Primary School in a Stockholm suburb where boys are seen entering a bus from the front and girls from the back.
Aged between six and ten, the pupils take the school bus in the mornings and evenings to go to and from school in the neighbourhood of Vällingby, north-west of Stockholm.
"I think this is despicable. This doesn't belong in Sweden," Prime Minister Stefan Löfven told reporters in Stockholm.
"We take the bus together here, regardless if you're a girl or a boy, woman or a man."
The school's vice principal said it had no intention of separating the children by gender.
"This is not something that has been known or sanctioned by school management. Both the principal and I note, after seeing the video, that it has happened, but it is of course nothing we support," Roger Lindquist told the TT news agency, saying that they would investigate why it had been done.
"I don't know why it still turned out that way," he added. "It was a mistake."
This isn't the first controversy surrounding the school. In August last year, Swedish media revealed that teachers had agreed to gender-segregated sports lessons. It argued then that gender-mixed sports courses would cause some parents to stop their children from attending.
The school describes itself as having a "Muslim profile", but is open to students from all backgrounds. Around 80 percent of the staff are non-Muslims, said Lindquist.
"It is important to point out that the staff is not driven by religion but a passion to work with cultural and integration issues. It is also important that the school is not based on Muslim values but democratic ones," he said.
Sweden's free school system of state-funded but privately run schools was introduced in 1992 and paved the way for religious organizations to operate schools as long as they stuck to the secular Swedish curriculum.
In August 2016 there were 66 religious free schools in Sweden, 11 of which were Muslim, according to the education ministry.