Police in Sweden's second largest city were called to the building at 8.24am after a man left a bag at the reception there.
“He said it would explode in ten minutes, then he turned and walked away,” Thomas Fuxborg, a spokesperson for the region's police force, told the TT news agency.
Officers quickly cordoned off the area, stopped traffic around the city hall and helped staff to leave the premises.
Between 80 and 90 people are understood to work inside the building.
At 9.20am witnesses reported hearing a loud bang, which was understood to be police detonating the suspected bomb.
But Fuxborg later told Gothenburg newspaper GP (Göteborgs-Posten): “We have shot at the bag with water. It did not trigger any explosion”.
Stefan Gustafsson, another police spokesperson told TT that officers had “quickly realized that the contents of the back were harmless”.
Police said that the man who left the bag appeared to be aged between 20 and 25, around 160cm tall and had dark skin. He was also wearing a blue hoodie and black sunglasses and ran away from the area in the direction of a nearby park, Brunnsparken.
By 10.30am officers said that they were questioning a man that fitted the description of the suspect.
Police outside Gothenburg city hall on Wednesday. Photo: Adam Ihse/TT
One of those who had to leave the building was the city's mayor Anneli Hulthén.
“We have responded in a calm and balanced way and I hope that everyone is evacuated now,” she told Aftonbladet at 9.30am.
Gothenburg's council was scheduled to meet at the city hall for the first time since its summer break on Wednesday morning, but it was unclear if the bomb threat was connected to the gathering, which has now been moved to another location.
“It may be a coincidence, but I dare not say,” added Hulthén.
Police have struggled to stop gang violence in the Swedish city in recent months, but officers did not immediately link the bomb threat to tensions between different groups in the area.