"There's no reason to protect our children from our traditions and our cultural heritage," Social Minister and Christian Democrat leader Göran Hägglund told the TT news agency.
Hägglund's comments come in response to guidelines issued by the National Agency for Education (Skolverket) designed to help school principals find a balance between Sweden's secular and religious traditions.
"The law stipulates that Swedish schools are non-confessional," the agency's Anna Ekström and Claes-Göran Aggebo wrote in an article in Dagens Nyheter (DN) on Sunday.
"That school is non-confessional means that there can't be any religious elements such as prayer, blessings or declarations of faith in education. Students should not have to be subjected to religious influence in school."
But Hägglund disagreed, arguing there is nothing wrong with having Swedish school children sing traditional hymns during Advent services or at graduation ceremonies, another annual occasion which prompts questions about how much religious influence should be allowed in Swedish schools.
"I'm tired of having this discussion two times a year. It has to be okay to sing a hymn in connection with something as solemn as a graduation ceremony," the minister told TT, going on to invoke a Biblical citation from the book of Matthew.
"The National Agency for Education is straining out gnats and swallowing camels."
According to Hägglund, Swedish students have bigger things to worry about than "singing O Holy Night one time a year".
"If the National Agency for Education now interprets the Education Act so strictly, I think we ought to consider changing the law and make an exception for these types of traditional ceremonies," said Hägglund.
Education Minister and Liberal Party (Folkpartiet) head Jan Björklund agrees.
"You should absolutely be able to sing hymns in church at school graduations; pastors should be able to read the Christmas gospel, refer to the Bible, and explain why we celebrate this Christian holiday," Björklund told TT.
He added that the agency's interpretation of the Education Act "set boundaries which are too strict", arguing that Advent services are "part of our cultural heritage".
"If this ridiculous debate continues year after year, I'm ready to review the Education Act," said Björklund.