The Swedish foreign ministry launched negotiations with authorities in Djibouti for the two Swedish nationals, both of Somali origin, to travel back to Sweden.
But before negotiations were concluded, the two men were taken to the United States to face charges of participating in weapons and explosives training with al-Shabaab during a four-year period beginning in 2008.
"If you can say that we proposed to have them come to Sweden and that they didn't come to Sweden, I guess you could call that a failure," foreign ministry spokesman Anders Jörle told Sveriges Radio (SR).
Meanwhile, relatives of one of the men have suggested that the Swedish foreign ministry may have been complicit in the still mysterious process that resulted in the two men, aged 27 and 29, awaiting trial in the United States.
The brother of the 27-year-old said his brother told him that American agents who interrogated the Swede said they were awaiting approval from the Swedish foreign ministry before extraditing him to the United States.
Jörle has said that the Swedish foreign ministry was not involved in the process, but was unable to confirm or deny whether or not Sweden may have given its approval to the extradition.
"I don't have any knowledge of that," he told SR.
According to family members, the two Swedes had initially travelled to Somalia to join al-Shabaab, but had later regretted the move and were attempting to leave the group.
While the news came as a relief to family members, they became increasingly concerned for the pair's safety.
"He said that he was very scared and had seen people be executed," Hassan Yusef, brother of the 29-year-old, told SR.
The 29-year-old's lawyer has previously said that the defendants have not committed any crimes against the US.
"American authorities believe that al-Shabaab, which has been designated as a terrorist group by the United States, propagates its activities on the internet and therefore 'affects American foreign trade'. In that case more or less anything falls within American jurisdiction," lawyer Ephraim Savitt told the TT news agency in December.
The men appeared in court in New York together with a 23-year-old Briton. They were aided by a Swedish interpreter.
If found guilty, they could face up to 30 years in prison.