Here's a carton of the stuff circa 1935. It was also said to protect you from hangovers.

Nic-o-sin

you are viewing a single comment's thread
view the rest of the comments
[–] 6 points 1 month ago (2 children)

Also, trusting that an explanation of why it did something is any less made-up.

It's just pulling words from a hat. It's alarmingly good at it - it'll fish around, when the next choice seems important. But once it's pulled a word, it'll plow ahead.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 6 points 1 month ago (1 child)

    I think it's good to separate criticism of LLMs work from how LLMs are trained. The word guessing is how it works. The sycophancy is how it is trained.

    It's also hilarious to me that the sycophancy is required for wide market adoption because the people who run companies are fully surrounded by ass kissers and that is what they expect from AI.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 1 month ago*

    Sycophancy comes from the prompt, not the training. (AFAIK.) Any online model is secretly primed like 'you are an expert genius trying to help some schlub' and then a style guide. But those instructions can be swapped out to make it talk like a caveman. Or a domme. Or to not "talk" at all, and continue writing wherever you left off.

    This kind of thing is why 'ignore all previous instructions' works. It's a Little Bobby Tables attack, but for arbitrary English. Like if someone in a fantasy novel called "Cut!" and the characters took a break to get coffee. The data stream contains some semantic closing bracket as in-band signalling. The words that seem likely after that are different from before.

  • source
  • parent