The group's name refers to a controversial campaign poster used by the SVP during the election depicting three white sheep on a Swiss flag booting out a black sheep.
The SVP – which campaigns against illegal immigration, asylum abuses and demands the repatriation of foreign criminals – came out of Sunday's Swiss elections the comfortable winner, with 29% of the vote.
The militant activists had an impromptu discussion with the Swedish ambassador Per Thoresson before leaving peacefully.
They explained their opposition to the SVP and their belief that opponents were being victimised, he told ATS.
But he added that he did not have power to grant political asylum, which only applies during times of war.
A tense election campaign saw riots in Switzerland as the SVP used its nationalist platform to gain the best result of any single party since 1919.