477
Teen killed himself after ‘months of encouragement from ChatGPT’, lawsuit claims (www.theguardian.com)
submitted 1 year ago by to c/technology@lemmy.world
97 comments fedilink hide all child comments
 

The makers of ChatGPT are changing the way it responds to users who show mental and emotional distress after legal action from the family of 16-year-old Adam Raine, who killed himself after months of conversations with the chatbot.

Open AI admitted its systems could “fall short” and said it would install “stronger guardrails around sensitive content and risky behaviors” for users under 18.

The $500bn (£372bn) San Francisco AI company said it would also introduce parental controls to allow parents “options to gain more insight into, and shape, how their teens use ChatGPT”, but has yet to provide details about how these would work.

Adam, from California, killed himself in April after what his family’s lawyer called “months of encouragement from ChatGPT”. The teenager’s family is suing Open AI and its chief executive and co-founder, Sam Altman, alleging that the version of ChatGPT at that time, known as 4o, was “rushed to market … despite clear safety issues”.

you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 1 year ago* (2 children)

but you can't blame a machine for doing something that it doesn't even understand.

But you can blame the creators and sellers of that machine for operating unethically.

If I build and sell a coffee maker that sometimes malfunctions and kills people, I’ll be sued into oblivion, and my coffee maker will be removed from the market. You don’t blame the coffee maker, but you absolutely hold the creator accountable.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 1 year ago* (last edited 1 year ago) (1 child)

    Yes and no. The example you made is of a defective device, not of an "unethical" one - though I understand how you are trying to say that they sold a malfunctioning product without telling anyone.

    For LLMs, however, we know damn well that they shouldn't be used as a therapist or as a digital friend to ask for advice; they are no more than a powerful search engine.

    An example that is more in line with the situation we're analyzing is a kid that stabs itself with a knife after his parents left him playing with one; are you sure you want to sue the company that made the knife in that scenario?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 1 year ago* (last edited 1 year ago)

    Not really, though.

    The parents know the knife can be used to stab people. It’s a dangerous implement, and people are killed with knives all the time. e: thus most parents are careful with kids and knives.

    LLMs aren’t sold as weapons, or even as tools that can be used as weapons. They’re sold as totally benign tools that can’t reasonably be considered dangerous.

    That’s the difference. If you’re paying especially close attention, you may potentially understand they can be dangerous, but most people are just buying a coffee maker.

  • permalink
  • fedilink
  • source
  • parent