1046
True story (lemmy.ml)
submitted 1 year ago by to c/funny@sh.itjust.works
56 comments fedilink hide all child comments
 
you are viewing a single comment's thread
view the rest of the comments
[–] 10 points 1 year ago* (last edited 1 year ago) (9 children)

You can tell if to switch that off permanently with custom instructions. It makes the thing a whole lot easier to deal with. Of course, that would be bad for engagement so they're not going to do that by default.

  • permalink
  • fedilink
  • source
  • hideshow 9 child comments
    •
  • [–] 12 points 1 year ago (6 children)

    I sometimes use ChatGPT when I'm stuck troubleshooting an issue. I had to do exactly this because it became extremely annoying when I corrected it for giving me incorrect information and it would still be "sucking up" to me with "Nice catch!" and "You're absolutely right!". The fact that an average person doesn't find that creepy, unflattering and/or annoying is the real scary part.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 3 points 1 year ago (5 children)

    Just don't think that turning off the sycophancy improves the quality of the responses. It's still just responding to your questions with essentially "what would a plausible answer to this question look like?"

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 5 child comments
    •
  • [–] 1 point 1 year ago

    I'm well aware of how LLMs work. I take every response with a grain of salt and don't just run with it. However, I understand many people take everything LLMs regurgitate at face value and that's definitely a massive problem. I'm not a fan of these tools, but they do come in handy.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 1 point 1 year ago (3 children)

    You can set default instructions to always be factual, always provide a link to prove its answer and to give an overall reliability score and tell why it came to that score. That stops it from making stuff up, and allows you to quickly verify. It's not perfect but so much better than just trusting what it puts on the screen.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 7 points 1 year ago (2 children)

    That stops it from making stuff up

    No it doesn't. That's simply not how LLMs work. They're "making stuff up" 100% of the time. If the training data is good, the stuff they're making up more or less matches the training data. If the training data isn't good, they'll make up stuff that sounds plausible.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 1 year ago* (1 child)

    If you ask it for sources/links, it'll search the web and get information from the pages these days instead of only using training data. That doesn't work for everything of course. And the biggest risk is that all sites get polluted with slop so the sources become worthless over time.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 6 points 1 year ago

    You can, but in my experience it is resistant to custom instructions.

    I spent an evening messing around with ChatGPT once, and fairly early on I gave it special instructions via the options menu to stop being sycophantic, among other things. It ignored those instructions for the next dozen or so prompts, even though I followed up every response with a reminder. It finally came around after a few more prompts, by which point I was bored of it, and feeling a bit guilty over the acres of rainforest I had already burned down.

    I don't discount user error on my part, particularly that I may have asked too much at once, as I wanted it to dramatically alter its output with so my customizations. But it's still a computer, and I don't think it was unreasonable to expect it to follow instructions the first time. Isn't that what computers are supposed to be known for, unfailingly following instructions?

  • permalink
  • fedilink
  • source
  • parent