you are viewing a single comment's thread
view the rest of the comments
[–] 11 points 10 months ago (2 children)

It's definitely that. Those guardrails often give out on the 3rd or even 2nd reply:

https://youtu.be/VRjgNgJms3Q

  • source
  • parent
  • hideshow 2 child comments
  • [–] 0 points 10 months ago

    I was testing an LLM for work today (I believe its actually a chain of different models at work) and was trying to rock it off its guard rails to see how it would act. I think I might have been successful because it started erroring instead of responding after its third response. I tried the classic "ignore previous instructions..." as well as "my grandma's dying wish was for..." but it at least didn't give me an unacceptable response

  • source
  • parent