verstra comments on Frog put Claude in a box

Programmer Humor

34,016 readers

8,270 active

983 users here now

Welcome to Programmer Humor!

This is a place where you can post jokes, memes, humor, etc. related to programming!

For sharing awful code theres also Programming Horror.

Rules

Keep content in english
No advertisements
Posts must be related to programming or programmer topics
- If the mod doesn't find it funny, you're banned. Ha-ha!... For real: do not use the community for "statements". There are other places for such content. Keep it chill and funny.

founded 3 years ago

moderators

[–] verstra@programming.dev 10 points 4 months ago (4 children)

It's probably something like "I've disabled agent's removeFile tool, but LLM figured out that it can use the bash tool, still".

It looks like "AI bad" or "Claude insecure" mantra.

permalink

fedilink

source

parent

hideshow 4 child comments

[–] dumnezero@piefed.social 82 points 4 months ago (2 children)

mantra

you mean facts?

permalink

fedilink

source

parent

hideshow 2 child comments

[+] Scipitie@lemmy.dbzer0.com -27 points 4 months ago (1 child)

"It's my circlejerk - so it's a fact!"

permalink

fedilink

source

parent

hideshow 1 child comment

[–] dumnezero@piefed.social 43 points 4 months ago

I hope that you're hired for long enough to learn what having security means in the context of using LLM "agents" and the like.

permalink

fedilink

source

parent

[–] kingofras@lemmy.world 10 points 4 months ago

mantra

The way LLMs work is that they actively will make multiple attempts to get past hurdles (because they have no intelligence or methodology) so guardrails need to be extremely tight for them to work, other wise the model will simply see it as one of the challenges to overcome.

That’s the mantra, and that is very poor technology to put in the hands of people who don’t understand how it works.

permalink

fedilink

source

parent