The ship, named Langeland, sent out a mayday at around 6am on Friday morning after it began listing in the stormy waters off the coats near the Swedish town of Strömstad, according to a statement by the Swedish Maritime Administration (SMA).
"This morning at 6am, we received a distress signal from the Langeland vessel indicating it had problems," said Dan Nicklasson, a captain with the Swedish coastguard in Gothenburg.
The coastguard has deployed 20 personnel in the search for the missing crewmen, who are from Russia and Poland, but Nicklasson admitted that "time was running out" for them to be found alive.
"No bodies have been found as yet. We have a helicopter and aircraft searching the area now. We found life rafts and wreckage from the ship. We also found patches of oil," added Christer Fjaellstroem, a duty officer for the Swedish coastguard.
The Langeland, which is 70 metres long and weighs 2,500 tonnes, was loaded with stone and was on en route to Moss in Norway. The sea rescue service has no further details on how much cargo the ship was carrying.
"It is probably the windy weather that is the reason behind the accident. The ship entered the Koster Fjord in search of shelter from the winds" Laila Svendsen at the service told the TT news agency.
According to the Swedish coastguard the Langeland has run into trouble off the Swedish coast on a prior occasion. In February 2007 it ran aground off Landskrona while carrying 30 cubic metres of oil.