The party wanted to pay the channel 1.5 million kronor ($201,240) to run the ad. The half-minute advert shows a race in which an elderly woman with a walker is chased by a group of burqa-clad women pushing prams with a slogan promising to safeguard pension funding at the expense of immigration.
As an alarm-like sound plays in the background, a voiceover says, "All politics are about priorities - now you have a choice."
The clip promotes the Sweden Democrats' demand that, like other parties, pensioners' taxes be cut to the same levels of wage earners. However, they claim their plans would be funded by reducing immigration.
"We decided not to broadcast it," Gunnar Gidefeldt, communications director for TV4, told AFP.
Swedish law on freedom of expression prohibits messages that contain hate grounded on race and religion, said Gidefeldt.
"In this case, it is against religion," he said.
According to party press secretary Erik Almqvist, the ad does not violate Swedish law. The party has screened the clip for lawyers, who said that it does not break the law against inciting racial hatred.
"The conflict we see as a result of mass immigration is not related to the person's origin, but rather a conflict of values, as far as we can see," said Almqvist in reference to the burqa-clad women in the video.
TV4 CEO Jan Scherman disagreed.
"The film is contrary to the democracy clause in the Radio and Television Act and also against democracy clauses which the Sweden Democrats, among others, have adopted for the equality of all people, regardless of whether it is the European Convention or the UN Charter," he said.
"The film is also against the constitution act on freedom of speech that prohibits hate speech," Scherman added.
Per Hultmangård, a lawyer at the Swedish Media Publishers' Association (Tidningsutvgivarna), came to a different conclusion. He does not see how the video would violate the law.
"I cannot see how this would be hate speech," he told news agency TT. "This is an election ad. The scope is wide for what one can say. They simply play on people's fears. Legally, it is within the allowable framework."
However, Scherman stood by his position and referred to an EU directive that is the basis for the wording of the Broadcasting and Television Act.
"The directive prohibits incitement of hatred according to race, sex or religion, which supports my decision," he said.
"It is quite clear to me as the editor responsible that those who watch the clip, together with the text, images and sound, very clearly see a group portrayed as intimidating and aggressive. The group is very easily identifiable, belonging to a religion, dressed in a certain way and attacking another group," he added.
According to Scherman, that group is the Muslims.
"There are probably lawyers and press experts who disagree on this," he said. "It is for TV4 and I to make an independent decision based on our knowledge, experience and perception of the law. It is not possible, even if one gets advice and opinions, to refer to them when making a editorial decision. It must be based on the conclusion that we and I have come to."
The Sweden Democrats could play kingmaker in the election on September 19th, which is up for grabs between the two coalitions. According to a survey from last Friday, just a month ahead of voting, the party was polling at 3.6 percent of the vote, just shy of the minimum of 4 percent that is required to enter Sweden's parliament, the Riksdag.
If they can reach this threshold for the first time, political analysts believe the party could be in a powerful position with the two main blocs on course to split the vote.