He is going to become Duke of Skåne, Sweden's southernmost region, the King said at a press conference in Stockholm on Thursday.
Also paying tribute to his own sister Princess Birgitta's husband who passed away on Wednesday, the King said: "Life is wondrous and in some peculiar way it goes on, that is what is so amazing."
A total of 34,762 men and boys are already called Oscar in Sweden, according to Statistics Sweden.
The King said he had not yet met his new grandson but would be seeing him "at a distance" later in the day, revealing that he had come down with a bout of influenza and did not want to pass on the bug.
But Prime Minister Stefan Löfven and Speaker of Parliament Urban Ahlin, who were also present at the press conference, said they had both met the child and assured media he was "exceptionally cute".
From left, Prime Minister Stefan Löfven, Prince Carl Philip and King Carl XVI Gustaf. Photo: Janerik Henriksson/TT
A 21-gun salute was fired from the Skeppsholmen island in Stockholm, Boden, Härnösand, Karlskrona and Gothenburg at noon in honour of the new prince.
It was Crown Princess Victoria's husband Prince Daniel who first revealed a baby had been born at a press conference at the Karolinska University Hospital in Stockholm late on Wednesday. He told Swedish media that his wife had given birth to a boy at 8.28pm, weighing 3,655 grams and measuring 52 centimetres.
“Everything's gone very well, it happened quickly,” he said, adding how impressed he was with his wife. “You women are a primal force. (…) He looks like his mother, which is pleasing.”
The proud father said he had been at the hospital during the birth and had cut the umbilical cord himself.
Prince Daniel trying to show how big the new baby is when he spoke to media on Wednesday. Photo: Henrik Montgomery/TT
Ahead of her visit, Prince Daniel told Swedish media: "Estelle is really enthusiastic and happy to become a big sister."
Princess Victoria joked last year that the little girl would “want a hamster” rather than a sibling.
However the Prime Minister Stefan Löfven revealed that Estelle had prepared some art work for her brother.
"She was a little disappointed that he had his eyes closed, because she wanted to show all the drawings which she had drawn," he told Swedish media, smiling.
The Royal Court's press office confirmed just after 9am that the family had left the hospital and had been allowed to return to their city residence, the Haga Palace.
"Both mother and child are in good health," read a statement in English on the court's website, which also issued a somewhat fuzzy photo of the royals heading home.
Sweden's Crown Princess Victoria, Princess Estelle and Prince Daniel, heading home after the birth of their new tiny family member. Photo: Royal Court
A more staged earlier snap showing Sweden's Crown Princess Victoria, Princess Estelle and Prince Daniel, who will welcome the new prince into their family. Photo: Kate Gabor/Royal Court
Victoria's two siblings, Princess Madeleine and Prince Carl Philip, were both spotted visiting the hospital on Wednesday evening to welcome the newest family member, who is third in line to the throne after his mother and sister. They also attended a special service on Thursday.
Meanwhile, Swedes and royal fans worldwide rushed to social media to share their joy about the birth, while others signed a congratulations book in Stockholm.
Swedish children signing a congratulations book in Stockholm. Photo: Anders Wiklund/TT
It's a boy! Congratulations #swedish Crown Princess Victoria and Prince Daniel on the birth of their 2nd child.- Sweden in Canada (@SwedenInCAN) March 2, 2016
If Sweden's new royal baby is even HALF as cute as Princess Estelle, it'll be SO. DANG. ADORABLE.- Diana Pearl (@dianapearl_) March 2, 2016
King Carl XVI Gustaf and his wife Queen Silvia now have four grandchildren, including London-based Madeleine's two children Leonore and Nicolas. The birth marks the start of a baby bonanza spring for the royals, with Carl Philip and Princess Sofia expecting their first child later this month or in April.