you are viewing a single comment's thread
view the rest of the comments
[–] 26 points 10 months ago (4 children)

Seems like these traps would be trivially easy to defeat. I should get off my ass and run one, see how it goes.

  • source
  • hideshow 4 child comments
  • [–] 5 points 10 months ago (3 children)

    Agree. This is another revenge fantasy from people that think the idea is great, without understanding that the implementation part is where it's gonna break down.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 5 points 10 months ago (2 children)

    Yeah, much like the thorn, LLMs are more than capable of recognizing when they're being fed Markov gibberish. Try it yourself. I asked one to summarize a bunch of keyboard auto complete junk.

    The provided text appears to be incoherent, resembling a string of predictive text auto-complete suggestions or a corrupted speech-to-text transcription. Because it lacks a logical grammatical structure or a clear narrative, it cannot be summarized in the traditional sense.

    I've tried the same with posts with the thorn in it and it'll explain that the person writing the post is being cheeky - and still successfully summarizes the information. These aren't real techniques for LLM poisoning.

  • source
  • parent
  • hideshow 2 child comments