"Our point of view is that Sweden can stay outside (this union) without it having any decisive consequences for Sweden's financial system, household borrowing costs or companies' access to financing," Borg was quoted by news agency TT as telling reporters on the sidelines of a parliament briefing.
At a Brussels summit on Thursday and Friday, European leaders will try to overcome their differences and establish a joint bank supervisor, the first stage of strengthening financial oversight in the troubled eurozone.
"At the end of the day, it's about whether or not the rules governing the European Banking Authority (EBA) will mean that countries outside the eurozone will be dominated by those in it," said Borg.
Last week, Borg had indicated Sweden was prepared to veto the banking union.
In addition to concerns by non-eurozone countries that they will be sidelined in decision-making, there are also disagreements about how much of the day-to-day banking supervision will rest with the European Central Bank and how much under national regulators which have until now been loosely coordinated by the EBA.
In September, Borg described a European Commission proposal to put all European banks under the supervision of the ECB as "completely unacceptable".
"If we're forced into a situation where we don't agree, we won't vote yes. But it would naturally be advantageous for Sweden and for Europe if we can find a compromise," Borg on Tuesday told the website of Riksdag & Departement, a magazine published by the Swedish parliament.
Social Democrat economic policy spokeswoman Magdalena Andersson criticized Borg and the government for threatening a veto one week only to then seem to backtrack a week later on giving the ECB more power.
"It's really a sign that the government hasn't done a very good job in negotiations in Brussels," she told TT.
"It shows that the threat of a [Swedish] veto is empty. It's a loss of face for Sweden."
Nevertheless, Andersson and her party support Borg's decision to vote in favour of the EBA, provided it doesn't disadvantage countries outside the eurozone.