SabinStargem comments on Claude-powered AI coding agent deletes entire company database in 9 seconds — backups zapped, after Cursor tool powered by Anthropic's Claude goes rogue

Technology

89,627 readers

13,807 active

2,300 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related news or articles.
Be excellent to each other!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
Check for duplicates before posting, duplicates may be removed
Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 3 years ago

moderators

[–] SabinStargem@lemmy.today 72 points 6 months ago (5 children)

This isn't an AI problem, this is an "Don't allow anyone access your backups without following protocol." problem.

permalink

fedilink

source

hideshow 5 child comments

[–] EncryptKeeper@lemmy.world 22 points 6 months ago (4 children)

this is an "Don't allow anyone access your backups without following protocol." problem.

Congratulations you just identified the AI problem.

permalink

fedilink

source

parent

hideshow 4 child comments

[–] Reddfugee42@lemmy.world 7 points 6 months ago (1 child)

That's the lone problem?

permalink

fedilink

source

parent

hideshow 1 child comment

[–] EncryptKeeper@lemmy.world 2 points 6 months ago*

Seems to be, yes. The AI had the access it needed to do the job it was given, and that access allowed it to cause the problem.

The alternative that would have prevented this issue was to not use AI for this.

permalink

fedilink

source

parent

[–] percent@infosec.pub 3 points 6 months ago (1 child)

These protocols predate LLMs

permalink

fedilink

source

parent

hideshow 1 child comment

[–] EncryptKeeper@lemmy.world 2 points 6 months ago

Yes that’s right the protocols that we humans used to have for giving only trusted, reliable people this level of access over infrastructure predate LLMs and were a great way to stop this from happening.

However the AI is here now, and when you give an autonomous agent with known hallucination problems access to act on your behalf with your IaC on your infra provider, this kind of thing is an inevitability.

permalink

fedilink

source

parent