When child protection services intervened, the 5-year-old son was spending hours simply cycling around and around his neighbourhood in Halmstad, southern Sweden.
His disabled 8-year-old sister was found tied to a chair, while the then 1-year old toddler was found sitting in her own faeces.
The baby has a Swedish father and has the right to stay on in Sweden.
Yet despite going through the entire Migration Board (Migrationsverket) appeals process, the two older children will not be allowed to stay. Their 26-year-old mother told Sveriges Television (SVT) that she looks forward to being reunited with her children.
The foster mother of the disabled daughter, however, likens the decision to a death sentence.
"I think in the long term this means she'll die, that's what's waiting for her," the foster mum told SVT.
The Migration Board, however, says it is not its job to make a judgment about whether the biological mother is a fit parent or not.
"There is still a very strong tie between the children and their mother," spokeswoman Annette Bäcklund said.
The Migration Board on Tuesday told the regional Göteborgs-Posten (GP) newspaper that it would review the case and see if there were adequate support structures for the children in Serbia.
"We'll get in touch with child protection services and look at their documents," spokesperson Fredrik Bengtsson told GP.