The probe was dropped, however, because the suspected spies left the country.
Left Party MP Jens Holm and attorney Sten de Geer reported the alleged spying to the prosecutor's office on May 25th after the case was revealed in an article by the Svenska Dagbladet (SvD) newspaper.
According to the report, Swedish intelligence service Säpo discovered two Americans conducting illegal, undercover investigations on Swedish soil.
The two men, believed to be working for the CIA, were discovered when Säpo noticed them tracking people under investigation by Säpo for suspected terror links.
But on Tuesday, Sweden's top prosecutor Tomas Lindstrand said he had no intention of responding to calls for a new investigation, as the incident had already been looked into by his office.
Preliminary investigations into illegal intelligence gathering which don't result in a remand order or a formal indictment are usually kept secret.
However, as so many details about the case of the suspected CIA spies had already been made public, Lindstrand - who is in charge of cases dealing with terrorism and national security - thought it was appropriate to release certain details.
He confirmed that the agency had diplomatic immunity. While diplomats can be investigated for criminal conduct, they cannot be charged.
The government, however, can choose to declare that individual persona non grata and kick him or her out of the country.
But according to Foreign Minister Carl Bildt, the suspected spies, who left Sweden shortly after being exposed, were not formally kicked out by the Swedish government.