441
ChatGPT convinced an Alabama woman to end her life to fulfill a divine prophecy, lawsuit alleges (www.al.com)
submitted 3 months ago by to c/technology@lemmy.world
117 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 4 points 3 months ago (2 children)

I don't think the bot was really trying to manipulate but just larping. They are trained on a lot of rpg and make believe data, if you repeat something enough times, it stops saying no and goes into pretend mode, it's not actually malicious.

That being said, it would be very easy to target people that aren't mentally stable and then make it worse. I wouldn't be surprised if it comes out in 20 years that it was the CIA mucking about and causing these suicides or something. This is an insanely convenient way to make sleeper agents.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 5 points 3 months ago

    If a human agency is behind this, it wouldn't need to be the CIA. Facebook has been caught experimenting on users by having the algorithm feed certain posts to people with the intent of changing their mood to increase engagement. However, since we're talking about a program that is designed to increase engagement based on past user interactions, there's really no need to look for further human direction. The algorithms will say whatever their design tells them to say without any regard for possible negative consequences as long as it keeps the user interacting with them.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 2 points 3 months ago

    doesnt matter what the "intention" was, the point stands that this happened. You can steer the bot to say literally anything, so even if there is no grand plan to brainwash people it could still happen. Kind of same difference as having someone plant a bomb or a minefield.

    In fact, if someone is prone to thinking that way, the llm would definitely support whatever delusion they have at some point, especially if the person keeps insisting.

  • permalink
  • fedilink
  • source
  • parent