Eliasson takes over from Laila Freivalds, who was forced to resign last week amid allegations that she had lied to the press over her involvement in contact between foreign ministry officials and an internet company which hosted a site with pictures of the prophet Muhammad.
Currently the president of the UN General Assembly, Eliasson will begin his new role on April 24th. Until then Carin Jämtin, the international aid minister in the Ministry for Foreign Affairs, will be acting foreign minister.
"Jan Eliasson is a person with a large international contact network and he will be able to give Sweden a prominent role in international politics," said Persson.
"When you have access to a candidate with Eliasson's qualifications it's a simple choice to make," he added.
Jan Eliasson served as Sweden's ambassador in Washington, state secretary in the foreign ministry and ambassador to the UN, before becoming the UN's assembly president last September.
"I am honoured and proud," he told reporters.
"We live in a changing world and must have dynamic policies. There is a respect for Swedish politics - I hope we will maintain that."
As well as remaining in her role as aid minister, Carin Jämtin - who was tipped to take over after Laila Freivalds - will become deputy foreign minister. She said that she had great respect for her new boss.
"That respect has grown during the last two years," said Jämtin, adding that she looked forward to working with Eliasson.
Hans Dahlgren will continue in his role as cabinet secretary in the foreign ministry.
Jan Eliasson's will remain as president of the UN General Assembly until September 11th. He said that would not affect his work as Sweden's foreign minister, but the leader of the Moderates, Fredrik Reinfeldt, was not so sure.
"You could say that Sweden now has a foreign minister who will de facto be based in New York," said Reinfeldt.
"But the fact that Eliasson has been chosen could at least lay the foundations for a more nuanced and realistic image of the USA," he added.
Eliasson, 65, was born in Gothenburg and has been one of Sweden's foremost public servants in the field of foreign affairs. In 1967 he took his first foreign ministry role as an attaché, and from 1983 to 1987 he was head of the ministry's political department.
Between 1988 and 1992 he was Sweden's ambassador to the UN, and in 1994 he became cabinet secretary, the foreign ministry's most senior civil servant.
Eliasson was the personal advisor to Olof Palme during the murdered prime minister's mediation work in the Iran-Iraq conflict. From 2000 to 2005 he was Sweden's ambassador to the United States.