The American-based Swede had the Englishman down in the first round, before Groves battled back to take an entertaining fight the full distance.
Two judges scored it 115-112 and 116-111 to the champion, the third judge giving it to Groves, 114-113.
Groves, 27, for whom it was a damaging third defeat in 25 fights, wasted no time in exiting the ring.
Jack, who improved to 20-1-1 and was making the first defence of his belt, said: “He's a hell of a fighter”.
“He was prepared to go 12 rounds and I should have got the KO, but I didn’t.
“I had him but got a little carried away. I wanted to knock him out,” he told Swedish newspaper Expressen after the fight.
“I felt much stronger than him. But he has a lot of experience; he was ranked number one in the world.”
“I will now take a vacation, go to Jamaica next week, and there are a lot of big fights for me now, a lot of big names.”
Sweden has long been ambivalent about boxing. When the 1912 Olympic games were held in Stockholm there was no boxing event as the sport was banned at the time.
After the ban was finally lifted, Ingemar Johansson from Gothenburg became the world heavyweight champion in 1959. Although several of his fights were in Sweden, his three landmark meetings with American Floyd Patterson – Johansson won the first one but lost the next two – were all in America.
Johansson later won the European heavyweight title in Gothenburg in 1962.
Worries about the sport's safety soon resurfaced, however, and boxing was banned again in Sweden in 1970. That ban wasn't lifted until 2006 – and even then there were restrictions on the nature of contests.
No professional fight can last longer than 12 minutes, a limitation that puts paid to the likes of Badou Jack defending their title on their home turf, as title bouts usually consist of 12 three-minute rounds.