LLM's don't operate in a vacuum anymore. They're given tools and access to systems.

An LLM could for sure shut down a database if you ask it to reduce costs and you give it database management tools. That's getting closer to the paperclip situation.