The 34-year-old, who played for the Gunners from 1998 to 2007, joined Shimizu from Celtic last September on an 18-month deal with his annual salary estimated at 60 million yen ($769,000).
But Shimizu announced on Tuesday that the contract, covering the 2012 J-League season due to start on March 10, had been terminated by "mutual consent."
The Swede said that "after a long thought-out process and discussions with my family and representative... I felt despite my feelings for Japan and utmost respect for my teammates and the coaching staff at S-Pulse, this was not the right project for me."
"Different ambitions combined with not having the opportunity to participate in the (Asian) Champions League 2012 edition were my main reasons to explore other options," he added in a statement released by Shimizu and his agent.
Shimizu finished 10th in the 18-team J-League first division in the last March-December season.
Ljungberg, who was capped 75 times and scored 14 goals for Sweden between 1998 and 2008, made 11 appearances in Japan, including eight J-League games and three cup matches, without scoring.
He went back to Europe after Shimizu lost to Cerezo Osaka in the Emperor's Cup quarter-finals on December 24. He has not since returned to Japan, saying he needed treatment on his left hamstring injured in the Christmas Eve match.
Shimizu president Yasuhito Takeuchi admitted there were "some differences in vision" which led to the Swede's departure.
"S-Pulse would like to compete in the 2012 season with players and staff who share common vision and goals," he said in a statement on the club's website.
Ljungberg was part of Arsene Wenger's Arsenal side that won the Premier League title in 2001-2002 and 2003-04.