Reinfeldt was speaking in response to the furore surrounding comments made by armed forces chief Sverker Göranson, who earlier this month claimed Sweden's defences would hold out only for a week if tested by an invading force.
Göranson's statements prompted reactions from far and wide, with Swedish security service Säpo launching an investigation last week into whether he had revealed classified information.
He has since gone on sick leave for exhaustion.
Defence Minister Karin Enström also drew criticism soon after Göranson's comments when she claimed that the level of preparedness was appropriate given the implausibility of an attack.
"There is no basis for the opinions expressed in the media and that makes this debate very strange," Reinfeldt said at a press conference, according to the TT news agency.
He added that the parliamentary parties enjoyed near consensus when it came to the defence budget.
Reinfeldt said an attack aimed solely at Sweden, on Swedish soil, was highly unlikely. The country did not need to focus on a hypothetical attack, he explained.
On Tuesday, opposition leader Stefan Löfven blasted the Prime Minister for referring to the armed forces as a "special interest".
"It is not a special interest, it's a national interest," he told TT, adding that the Prime Minister's decision not to take action was "worrying".
"One of the government's most important tasks is to ensure we have good defences," he said.