A member of the public has now reported him to the police for breaking the Swedish law that protects the peace of the dead (brott mot griftesfriden).
The piece in question is being shown at a gallery in Lund.
In the text alongside the work, artist Carl Michael von Hausswolff explained that he collected the ashes from a concentration camp crematorium in Majdenak, Poland.
Years later, he mixed the stolen ashes with water and used the result as paint on acrylic paper.
The work has sparked a range of reactions from critics, with many upset that von Hausswolff would use remains from Holocaust victims in his art.
"You shouldn't use remains as souvenirs," art critic Tor Billgren told The Local.
Billgren's main objection is related to disturbing the peace of the dead (brott mot griftesfriden), rather than the specific oeuvre's link to the Holocaust.
Author and translator Salomon Schulman, however, reacted with more virulence.
"It is repulsive in the extreme," he told Sveriges Television (SVT).
In a stinging critique published in the local Sydsvenskan newspaper, Schulman also likened the work to a fascination with necrophilia and questioned whether it should be called art.
"Who knows, some of the ashes might come from some of my relatives?" he wrote, adding that he refused to see the work for himself.
Billgren expressed considerable respect for the rest of the exhibition at Lund's Martin Bryder Gallery.
"When von Hausswolff work isn't muddied by naivety he offers exciting and alternative strategies to deal with and remember the horrors of the past," Billgren wrote in his review of the show.
The gallery owner, Martin Bryder, has said it was never his intention to hurt anyone's feelings.
"This wasn't meant to open old wounds, rather it was meant to heal them," Bryder told SVT.
Von Hausswolff is a well-respected artist, once referred to as both "dour and funny" by the New York Times, and has represented Sweden at the Venice Biennale.
"I think Carl Michael von Hausswolff's art is interesting, but in this case he showed bad judgement," Billgren told the Local.
As news broke on Thursday afternoon that the artist had been reported to the police, the TT news agency was unable to reach him for comment.