Vilks, the Swedish artist who enraged Muslim groups with his depictions of the Prophet Muhammad as a dog, agreed earlier this month to speak at a conference organized by the anti-Muslim group Stop Islamization of Nations (SION).
Last week, however, an art gallery in northern Sweden booted him from a planned group exhibition because of his decision to accept the SION invitation.
Several of the other participating artists pulled out in solidarity with Vilks, stirring up a debate in Sweden this week about artistic freedom and Islamophobia.
The art exhibition, which was due to open on September 30th, has since been cancelled.
In a lengthy interview published in the Aftonbladet newspaper on Thursday, Vilks defended his decision to speak at the SION event.
"If the Ku Klux Klan had invited me, I would have gone," he told the paper.
In the lengthy Aftonbladet interview, Vilks said that SION is a "demagogic" organization that "cannot be disregarded".
"They have a growing political influence. Their campaigns generate violent attention," Vilks said.
SION, an international network of counter-jihadist groups, was formed in January 2012. The group's inaugural summit, which Vilks is due to speak at, will take place in New York on the eleventh anniversary of the 9/11 terror attacks.
SION's president, the controversial author and blogger Pamela Geller, has warned of the "Muslim agenda to grind away at our liberties until America is under Sharia law".
She once suggested that Malcolm X was President Obama's father and called him a "jihad president".
Vilks was invited to the SION summit to speak about the global storm caused by his 2007 drawings showing the head of the Prophet Muhammad on a dog's body.
Speaking to Aftonbladet, he explained that he is participating in the conference as an artist and that his visit is part of an ongoing art project.
"I'm obviously going to say that I'm there as an artist. That it's a part of my art project. 'You are a work of art'," he said.
Vilks has received a series of death threats and is currently under round-the-clock surveillance by the Swedish Security Service.
However, he explained that he does not see Muslims as a threat.
"Islam doesn't stand a chance against modernization," he said.
"I regard it as a temporary hysteria."
Speaking about immigration in Sweden, Vilks said he thinks that it's too large in relation to the resources".
"I don't have any opinion on how many people are let in, but they must be taken care of. People think I hate Muslims and immigrants, but I don't have a problem with people from other countries," he said.
Vilks also lamented official Sweden's approach to multiculturalism as "unbearable".
"You're supposed to celebrate Midsummer, but then you can't because you need to have other groups represented so that we don't appear to be racists," he said.
"But multiculturalism isn't something that can be stopped. It comes with internationalization automatically," he told Aftonbladet.