you are viewing a single comment's thread
view the rest of the comments
[–] 8 points 1 year ago (2 children)

From my understanding its length of the conversion that causes the breakdown. As the conversation gets longer the original system prompt that contains the guardrails is less relevant. Like the weight it puts on the responses becomes less and less as the conversation goes on. Eventually the LLM just ignores it.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 2 points 1 year ago

    Been tempted to fuck with ChatGPT, see what I can push it to say, but I don't want that on my "record". If it was utterly private, I'd be pushing the envelope. Be interesting to experiment with!

  • source
  • parent