The man was sitting in a car in the Lövgärdet suburb at around 09:00 on Thursday when several shots were fired at him by two people, who then left in a car driven by a third person.
The victim was pronounced dead by ambulance staff at the scene, and police have launched a preliminary investigation into murder. Several witnesses, including school students, saw the incident.
"It happened at Vaniljgatan, in a small centre you could say. There are several shops and apartments in the same area. We have a lot of witnesses and many who saw important details," police spokesperson Christer Fuxborg told news agency TT.
The victim is a man in his 30s who lives in the local area and was employed at a nearby school. The crime scene has been cordoned off while a forensic investigation is carried out.
Police are using a helicopter to search for the suspected perpetrators.
"It's a small blue car that left the area hastily, one driver sat in the car while two other people were busy out in the area," Fuxborg explained.
An update from the police on Thursday afternoon explained that they have impounded a vehicle that may be linked with the incident, but that is not yet certain. One person has been taken in for questioning, but that person is not under arrest or a suspect.
This is the first lethal shooting in Gothenburg in 2017, though in recent years the city has experienced problems with violent crime between rival gangs. Last year six people were shot to death in the greater Gothenburg region.