Archived version

Google’s AI search told users that an Aachen doctor had been convicted of a sex crime. The conviction belonged to a different man, and on September 24 the Aachen Regional Court [in Germany] banned Google from repeating it, ruling that the company answers for what its AI Mode says. It is the second German court this year to treat Google’s AI answers as Google’s own words, and the first ruling to say so has now been settled rather than tested on appeal.

...

The doctor asked Google to delete the text on August 28 and followed up with a formal legal warning on September 15. Neither worked. The court’s 8th civil chamber then acted without hearing Google first, citing the urgency of false claims that were “publicly visible” to an unlimited audience. Google Ireland, which runs Google Search in the EU, now faces a fine of up to €250,000 for each breach, or detention of its managers, if it repeats three versions of the text. Google can still challenge the order.

The court’s reasoning is blunt: Google “provides AI-generated search and is therefore also responsible for the answers generated in this context.” It added that checking would have been easy: had Google “confronted its own AI” with the doctor’s objections, it would have found the statements were false.

...

That reasoning comes almost word for word from Munich. On May 28 the Munich Regional Court I ruled (case 26 O 869/26) on AI Overviews that told searchers two Munich publishing companies were “known for dubious business practices”, ran subscription traps and kept changing their names to avoid being traced. None of the linked sources said that; the first one cited did not mention the companies at all.

Google argued that it was only summarising other people’s pages, the same defence that has long protected search engines in Germany. The court disagreed. AI Overviews, it said, do not just show links but make “independent, new and substantive statements” built by evaluating and combining third-party sites, so Google must treat them as its own, including as the provider of an AI system under the EU’s AI Act. “Only it has influence over the AI,” the judges wrote.

Google said at the time it would appeal. That appeal has ended in a settlement, The Decoder reported on Saturday; the terms are not known and neither Google nor the publishers would comment. The upshot is that no higher court has ruled, so the Munich judgment is not binding precedent, but lower courts such as Aachen are already following it.

...

all 12 comments

sorted by: hot top controversial new old
[–] 42 points 3 weeks ago (3 children)

This seems like a pretty obvious ruling, hopefully it is more broadly applied.

The 'oopsie my AI did it' excuse is wearing thin when 'it' is things like this, or encouraging teens to commit suicide. The company is responsible for what communications they send and should be liable for the damage that they cause.

  • source
  • hideshow 3 child comments
  • [–] 13 points 3 weeks ago

    I really dislike how much hype i read in the really pro ai spaces who think law is the next big field after medicine. We already have errors with alrps and we are already trying to introduce another way the justice system can introduce more errors. Lawyers are already getting in trouble with ai generated work that has obvious errors in it to things that don't exist. If that happens the case should just get thrown out of court, and the lawyers punished heavily.

    Like seriously are we going to make ai the judge, the jury, and the executioner too?

    People could argue that a person has a final say over it so that makes it okay. There's a soldier in Israel that reads an ai generated report that assures him that there is a terrorist in a building that will murder his family, he presses a button and bombs the building, and he has the final say over that too. Just numbers on a screen and they where probably bad people anyways, just following the book/ai.

    We are getting to the point where we are putting peoples lives at risk, or the very least a traumatic time with guns pointing a gun at your face accusing you of stealing your own car and a few days in the holding cell. Its crazy how much here in the united states ai gets free leeway to break anylaw possible, and some how its okay because an ai did it. They're rigging your rent prices, and getting away with manipulating prices uber drivers get in a way that is highly illegal if a human was involved yet its completely fine. Predictive markets are some how not gambling because ai. There needs to be a line, it is absolute insanity that trump made it illegal (essentially) for the states to make any law regulating ai in any shape or form. We literally are using the same tech they use to target and genocide kids its crazy.

  • source
  • parent
  • [–] 6 points 3 weeks ago (1 child)

    And the penalty needs to be steep enough that it's not just an operating cost for these companies.

    Google might just budget a few million a year for paying the fine every time a court rules they pushed someone to kill themselves if it's profitable enough.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 6 points 3 weeks ago

    EU countries have the right idea for fines, companies get fined as a percentage of their global gross revenue. 10% in some cases (and 20% if there is a repeat offense in 8 years).

    Having a few million dollar fine isn't going to create a proper deterrence.

  • source
  • parent
  • [–] 29 points 3 weeks ago (1 child)

    I don't know about Germany but in the UK repeating a libel is also libel, so "I was just repeating what someone else wrote" isn't a defence.

  • source
  • hideshow 1 child comment
  • [–] 12 points 3 weeks ago (1 child)

    Google argued that it was only summarising other people’s pages

    So they're admitting to plagiarizing? Either they are plagiarizing or they need to take responsibility. Pick one.

  • source
  • hideshow 1 child comment
  • [–] 8 points 3 weeks ago

    I hate that one has to travel halfway across the world from the US to see some semblance of a justice system doing the obviously correct thing.

  • source
  • [–] 7 points 3 weeks ago

    Google's like "Whaaaaat come on we even have that little text at the bottom:"

    "Results are probably false."

    "How could this happen??? Not our fault."

  • source
  • [–] 6 points 3 weeks ago

    I’m glad countries in EU take action like this, they still have balls. In U.S.A. Google instructs the court to ignore these things and the courts obey the orders from their masters.

  • source
  • [–] 2 points 3 weeks ago

    Well well well.

  • source
  • 50 comments in c/programming@programming.dev view that post
    [–] 126 points 3 weeks ago* (last edited 3 weeks ago) (22 children)

    For a company earning $403 billion in annual net income, €250,000 is just another budgeted operating expense. Fines should always be based on annual worldwide revenue.

  • source
  • hideshow 22 child comments
  • [–] 42 points 3 weeks ago (6 children)

    Detention of its managers might have more impact.

  • source
  • parent
  • hideshow 6 child comments
  • [–] 43 points 3 weeks ago (9 children)

    Eh, when you consider how often LLM's spit out wrong answers it could add up quick.

  • source
  • parent
  • hideshow 9 child comments
  • [–] 32 points 3 weeks ago (4 children)

    Is that supposed to be a downside? Seems like the consequences of not producing accurate and fully tested products that have the potential for massive negative effects

  • source
  • parent
  • hideshow 4 child comments
  • [–] 20 points 3 weeks ago (3 children)

    No, it is not a "downside", it is why this court ruling matters.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 17 points 3 weeks ago (2 children)

    Fines should actually impact companies, not just be another cost of doing business. By imposing a high dollar value fine per violation, regardless of frequency of violation, companies may actually be incentivized to stop breaking the law in order to continue existing

  • source
  • parent
  • hideshow 2 child comments
  • [–] 20 points 3 weeks ago

    There's a difference for a one-time fine that covers continued wrong doing over a period of time, and a fine about a specific instance of misbehaviour.

    The first don't accrue, and I agree, should be based on revenue so that they serve as actual punishment and serve as future deterrent.

    But a €250k that can be slapped by thousands of cases that happen every day (AI providing wrong information) has the potential to be massive as well.

  • source
  • parent
  • [–] 15 points 3 weeks ago* (last edited 3 weeks ago)

    I think the point other people are making is that this can set precedent in German law and allow other people with similar issues to bring suit and win on merit and not have to fight as hard for their case to be considered. It could also end up inspiring similar cases in other countries using similar frameworks, adapted for local laws.

    If suddenly Alphabet has thousands people taking to them to court in jurisdictions all over the world, that can indeed hurt them, although it will take longer than injunctions on Google now, I fully agree.

  • source
  • parent
  • [–] 2 points 3 weeks ago

    Make it a recurring subscription instead.

    10% of yearly revenue global for making a machine that lies (in a legally-compromising way) once per year. You want a lie per month? That will be 120% of your revenue for that year.

    The numbers can be tweaked, but it should not be tolerated. Were I to misbehave and say what Google's "AI" said, I would be in much more trouble than that.

  • source
  • parent
  • load more comments (-1 replies)
  • [–] 1 point 2 weeks ago

    That appeal has ended in a settlement.

    The fines, if even paid, were probably less than the settlement. That being said, the best case would have been going to court for precedence, but you can't blame someone for cashing out if they've been a victim.

  • source
  • parent
  • [–] 57 points 3 weeks ago (12 children)

    Why is anyone listening to AI? We were told not to use Wikipedia for reports....wtf

  • source
  • hideshow 12 child comments
  • [–] [S] 43 points 3 weeks ago* (last edited 3 weeks ago) (4 children)

    I fully agree, but the thing here is this doctor who was accused of a sex crime he didn't commit. They can't simply throw out a chatbot without a human being to be held accountable. This is one of the major threats of LLMs in my opinion: the danger isn't the LLM itself, but its creator's attempt to reject any human responsibility.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 15 points 3 weeks ago (1 child)

    Its the same as the SaaS revolution. The accountability is outsourced and everyone can just shrug. Same as police violence. Homeless camp gets disrupted and now there is a flower patch...huh. If no one is responsible, than no one has to be responsible.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 9 points 3 weeks ago (2 children)

    We were told not to use Wikipedia for reports

    Yeah maybe in 2003.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 9 points 3 weeks ago

    I mean, I have to say, recently Qwant introduced default-on AI responses and one reason I hate it, is that it shows up where they also show excerpts from Wikipedia etc.. It has happened to me that I was reading the generated text thinking it's an excerpt.

    I would not be surprised, if tons of users never realize that it's generated.

  • source
  • parent
  • [–] 9 points 3 weeks ago (1 child)

    Because wikipedia was never backed by billionaires owning virtually everything

  • source
  • parent
  • hideshow 1 child comment
  • [–] 8 points 3 weeks ago

    If Google was updating Wikipedia and falsely accusing somebody of sex crimes on their Wikipedia pages, I think you could expect that Google would get sued in that situation, as well. It's really about the source of the information.

  • source
  • parent
  • [–] 50 points 3 weeks ago (1 child)

    The sooner we make companies responsible for their products the better.

    Honestly there should be a lot more Brian Thompsons by this point.

  • source
  • hideshow 1 child comment
  • [–] 40 points 3 weeks ago* (last edited 3 weeks ago) (4 children)

    ChatGPT can't discuss Brian Hood because he sued them (it shuts down if you bring him up).

    We should all have the option to opt out of being discussed by LLM models. In fact, it should probably be an opt-in process. Not being discussed should be the default.

  • source
  • hideshow 4 child comments
  • [–] 23 points 3 weeks ago (2 children)

    Forgot about that! Just tried it.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 2 weeks ago*

    who is brian hood

    Depending on the context, Brian Hood most commonly refers to one of two notable figures: […]

    It tells me about two Brian Hoods. I assume one of them is the supposed denied one?

    Or it is selective about what information it gives about them.

  • source
  • parent
  • [–] 24 points 3 weeks ago

    Should have been very obvious a long time ago that you can't just publish wrong information, no matter how new and exciting your technology is...

  • source
  • [–] 17 points 3 weeks ago

    These kinds of AI output should not be protected by anything like Section 230. It is their own words and they should not get a pass for what essentially amounts to libel. Google needs to be reeled in. The fines have been too small and the damage has been too great. This isn’t even the first time something like this has happened. They have made up claims like this against plenty of other people. Claiming that they get a pass because it is AI is just using the LLM, their own LLM, as a scapegoat.

    Other examples https://www.theguardian.com/music/2026/may/05/canadian-ashley-macisaac-fiddler-musician-singer-songwriter-sues-google-ai-sex-offender-ntwnfb

    https://londondaily.com/robby-starbuck-sues-google-after-its-ai-made-up-totally-fake-rape-and-crypto-fraud-claims-against-him-and-supported-it-by

    Slightly different but related https://www.eff.org/deeplinks/2022/08/googles-scans-private-photos-led-false-accusations-child-abuse

  • source
  • [–] 11 points 3 weeks ago

    When there is a legal issue "it is only an advanced search engine". When it is time for PR "It is already more intelligent than most humans"

  • source
  • [–] 10 points 3 weeks ago

    In 2026, an AI on crack has accused a man of a crime he didn't commit

  • source
  • [–] 6 points 3 weeks ago (1 child)

    The upshot is that no higher court has ruled, so the Munich judgment is not binding precedent, but lower courts such as Aachen are already following it.

    This sentence sure does read like the author is glad the judgement isn't binding, as opposed to lower courts following it.

  • source
  • hideshow 1 child comment
  • [–] 1 point 2 weeks ago

    Wake me once Goofle has paid the fine and decides to stop giving AI summaries in Germany

  • source
  • 8 comments in c/technology@lemmy.world view that post
    [–] 37 points 3 weeks ago (1 child)

    Google Ireland, which runs Google Search in the EU, now faces a fine of up to €250,000 for each breach, or detention of its managers, if it repeats three versions of the text. [own emphasis]

    Holy shit that judge isn't fucking around.

  • source
  • hideshow 1 child comment
  • [–] 21 points 3 weeks ago

    Here is a fun game to try with your friends... flip the words of Company does X to random person does X ... then see what punishments feel appropriate. Company X destroys the 10% of the amazon becomes Todd from work destroys.. or Bob framed another person for sex crimes. Crazy how privileged companies are vs individuals.

  • source
  • [–] 18 points 3 weeks ago

    The "independent, new and substantive statements" framing is the part I think travels furthest. Google's defense was basically "we're just summarizing," but a summary that recombines facts into a new claim — this doctor was convicted — is authorship in any meaningful sense, so the old search-engine safe-harbor logic doesn't really apply. If that reading holds across the EU it gets interesting fast for every company bolting an AI chatbot onto their search or docs. It'd be cheaper to cite sources verbatim with links than to defend a generated claim in court, which honestly would also make the products better. Also notable this is the second German court to land on the same side this year — one ruling is an outlier, two starts looking like a direction.

  • source
  • [–] 7 points 3 weeks ago

    It’s about time …

  • source
  • [–] 13 points 3 weeks ago (2 children)

    Google argued that it was only summarising other people’s pages, the same defence that has long protected search engines in Germany. The court disagreed. AI Overviews, it said, do not just show links but make “independent, new and substantive statements” built by evaluating and combining third-party sites, so Google must treat them as its own, including as the provider of an AI system under the EU’s AI Act. “Only it has influence over the AI,” the judges wrote.

    It doesn't summarize, to summarize something you first need to understand what it says and even what it means.

    AI in it's current form doesn't understand the meaning of anything, just the statistical chance of what the next word should be.

    Al it's doing is paraphrasing, shuffling words around just enough to claim it doesn't owe the original writer anything.

    Most of us learned why that was unacceptable in jr high English; it's plagiarism if done correctly and nonsense if not.

    Google and other AI corps want to play it both ways, they want to reuse information for free while not being accountable for what it says. If they're not accountable because it's not their information, why aren't they paying the people who created the information?

  • source
  • hideshow 2 child comments
  • [–] 2 points 3 weeks ago

    Whether it has any sort of real understanding or not, it does summarise. It produces summaries as well as many other things. It's just very unreliable, due to a number of shortcomings that might just be fundamental: no real world model, no ability to distinguish between truth and lies, fact and fiction, except statistically, which sometimes results in nonsense. It can misrepresent, leave things out, and sometimes just make stuff up.

    It does summarise, but it does it poorly and unreliably.

  • source
  • parent