The Security Service (Säpo) will now take over the investigation.
"They have not yet been ruled out of the investigation, but it is the prosecutor's view that the suspicions regarding them are not sufficient for them to be charged," said Gothenburg police in a statement.
"The police think that there is no longer a threat of attack against the centre of Gothenburg," police said.
Police had arrested four men Saturday morning in one of the city's suburbs on suspicion of preparing a "terrorist crime", releasing two of them later the same day.
A report in the Swedish newspaper Aftonbladet, citing anonymous sources, said all four suspects were married men who were members of the same family and of Syrian origin.
The paper suggested that the suspects had been questioned in connection with the controversy over the 2005 publication by Danish newspaper Jyllands-Posten of 12 caricatures of the Prophet Mohammed. That incident sparked violent protests in the Muslim world the following year.
Swedish police said Friday they had been tipped off "by a credible source" about plans for a bomb attack in central Gothenburg, but security patrols there found nothing.