you are viewing a single comment's thread
view the rest of the comments
[–] 22 points 8 months ago (3 children)

Oh they're trying. Experiments have made so many mistakes.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 8 points 8 months ago (2 children)

    I saw a story recently where a guy spent some time with a customer service chatbot, and ended up convincing it to give him 80% off, and then ordered like $6000 of stuff.

    LLMs just don't produce reliable/predictable output, it's much easier for the user to get them to go off the rails.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 2 points 8 months ago (1 child)

    Aren't there also tons of studies and math that show/prove they cant differentiate between instructions (e.g. from the company) vs data (e.g. that guy's messages)?

  • source
  • parent
  • hideshow 1 child comment
  • [–] 2 points 8 months ago

    Yes, I believe that is the case.

    Of course in any other application, keeping instructions and data separate is very important. Like an SQL injection attack is when you're able to sneak instructions in where data is supposed to go, and then you can just delete the entire database, if you want. But with LLMs the distinction doesn't exist.

  • source
  • parent