"Role tags were a formatting trick that became the security architecture and the cognitive scaffolding of modern LLMs," the authors explain in a blog post. "We've shown that this architecture doesn't survive into the model's actual representations, and that such role confusion is linked to prompt injection."

you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 4 months ago

I thought they had synthesized the flavoring! Kinda cool, but also kinda weird that they're now coke dealers in both senses (again) lol.

  • source
  • parent