"The most probable outcome is that you end up with some kind of default in Greece," Borg said in an interview with Sveriges Radio (SR).
"Considering the way the situation has been handled so far, one cannot rule out that (Greece) will fall into bankruptcy in the end," he added.
Greek Prime Minister Antonis Samaras, who leads a new three-party coalition government, was meeting Thursday with auditors from the European Union, the International Monetary Fund and the European Central Bank in a bid to renegotiate his country's second bailout.
The Greek government is expected to promise creditors that it will redouble privatization efforts to show that it remains on a recovery path that will enable it to rejoin the world economy.
Borg stressed that even if the new Greek government manages to get an extension of payment deadlines and meets all its pledges, its target of a debt to gross domestic product ratio of 120 percent by 2020 remains too high.
"A 120 percent debt level is rarely sustainable, particularly not in a country with very weak growth prospects," Borg said.
The Swedish finance minister, whose country is not a member of the eurozone, meanwhile said it remained unclear if a default would necessarily push Greece out of the common currency bloc.
"Obviously, they only have harsh alternatives," he said.
"They could either do this by building up better prospects for the future inside the euro, or at the end of the day, they might choose to leave," he added.