The man, who hears cases in a district court in Skåne, in the south of Sweden, will not face a trial as he has agreed to pay a fine of 21,000 kronor. He is not expected to lose his job.
The judge was one of 67 men suspected of being customers at the brothel, which presents itself as a solarium, following a police raid in February.
So far 37 men have admitted using the place for more than its sun beds, but at first the judge told police he simply went there for a back massage.
"But when investigators played back the calls from the tapped phones, the judge suddenly began to remember," wrote Expressen, the Swedish tabloid which broke the news.
The call revealed him ordering sex with three girls, while he also admitted to police that on another occasion he masturbated while a prostitute watched.
The chief prosecutor, Sven-Erik Alhem, described the case as a "catastrophe for the confidence in the judiciary".
"Very unfortunate. It is deeply worrying when there are many cases. It can lead to a lack of trust in all of us who work in the judicial system," he told Expressen.
Just a few weeks ago a Swedish Supreme Court judge admitted to paying a 20-year old rent boy for sex. He did not lose his job but was moved sideways into a position on the Council on Legislation, a body which provides opinions on law proposals before they are submitted to parliament. He was fined 42,250 kronor.
In May of last year another Skåne judge was fined 39,200 kronor after being caught using a prostitute.
Sven-Erik Alhem told news agency TT that the chancellor of justice created an unfortunate precedent last month when he decided that the Supreme Court judge would not be fired.
But the chief judge at the court where the latest miscreant works has expressed his support.
"He's a talented judge and will continue in service just as before," he said.