18
Security researchers tricked LLMs into giving them cocaine recipes by abusing role models for prompt injection (www.theregister.com)
submitted 4 months ago by to c/enshitification@slrpnk.net
1 comment fedilink hide all child comments
 

"Role tags were a formatting trick that became the security architecture and the cognitive scaffolding of modern LLMs," the authors explain in a blog post. "We've shown that this architecture doesn't survive into the model's actual representations, and that such role confusion is linked to prompt injection."

other discussions
36
Security researchers tricked LLMs into giving them cocaine recipes by abusing role models for prompt injection (www.theregister.com)
submitted 4 months ago by to c/fediverse_vs_disinfo@lemmy.dbzer0.com
6 comments fedilink hide all child comments
 

"Role tags were a formatting trick that became the security architecture and the cognitive scaffolding of modern LLMs," the authors explain in a blog post. "We've shown that this architecture doesn't survive into the model's actual representations, and that such role confusion is linked to prompt injection."

16
Security researchers tricked LLMs into giving them cocaine recipes by abusing role models for prompt injection (www.theregister.com)
submitted 4 months ago by [B] to c/pulse_of_truth@infosec.pub
comment fedilink hide all child comments
 

If you want a picture of the future of LLM security, imagine Whac-a-Mole meets Groundhog Day