No injuries were reported in the incident, the cause of which was not immediately known.
The accident occurred around 2am, just minutes after the vessel had completed an exercise with three other ships near the island of Utö in the southern part of the archipelago.
“After a few minutes one of the boats began to take on water and began to sink. The captain quickly decided that the ship must be abandoned, the life rafts were put in the water and all 16 men left the ship,” the military said in a statement.
“Things went quickly and the crew ended up in the water. Crew on board the three other ships immediately began a sea rescue and rescue authorities were alerted. All 16 were rescued from the water,” it said.
Rough seas were reported in the area at the time, according to the military.