"You might say that the crisis has been caused to a very large extent by people not respecting the rules that are already there ... if they don't abide by the previous rules it doesn't make much sense to impose new rules," Bildt said during a visit to the Latvian capital Riga.
"But we do see the need to strengthen economic surveillance," he added.
"We need more Europe, but not more Europes," Bildt said, speaking out against proposals for a so-called "two-speed Europe" with a core of more-integrated states in the 27-member bloc based on European Union powerhouses Germany and France.
The Swedish foreign minister made the comments following talks with Latvian Prime Minister Valdis Dombrovskis.
After meeting with his Latvian counterpart foreign minister Edgars Rinkevics, Bildt further developed the theme.
"In some respects we have a twin-speed Europe. We have the growing economies of northern Europe and the indebted economies of southern Europe.
That is the way it is at the moment," Bildt said.
"We are very much in favour of preserving the institutional architecture that's necessary for the maintenance of the single market.
"When I said this quip about multiple Europes, that can be seen as a slight warning that we don't go into institutional set-ups that over time might impair the single market," he said.