I can run Bonsai, basically a distilled version of Qwen 3.5 (Qwen being the other major family of open weight Chinese developed LLMs) on my Steam Deck.

It ain't fast, but its kind of nuts that it can run at all on hardware that is that underpowered, by typical LLM hardware standards.

You just run the thing in a sandbox of some kind, do not give it access to the outside internet or the rest of your computer, and the worst thing that is likely to happen is ... you configure it slightly wrong, it uses too much RAM, and your system crashes. And your system can then reboot again, just fine, tone it down a bit to use less RAM.

For a large organization with networked security concerns and consistent uptime requirements, it is simply insane to just plug an LLM directly in to that, without architecting an entire system built around 'assume the LLM will fuck up/go insane'; sanboxes within sandboxes with highly controlled connections to other sandboxes.

But that would be a significant feat of actual software engineering to construct that for/around an actually complex network, and would take time, so fuck that, we want line go up is what most companies decided to do instead, just fucking raw dog it.

But yeah the main 'problem' with LLMs in this sense is idiot CEOs just demanding the magic machine god be plugged right in to their very fragile, barely sensible networks and workflows, where the LLM is then going to break things in roughly the same way as would the nepo hire intern who was just directly promoted to sysadmin...

...not that an LLM has a secret activation phrase that will cause it to shut down or go berserk in a specific way.

Nobody runs it on their own hardware though (which costs tens of thousands of dollars). I'm willing to bet $100 that government institutions that were using it were just using cloud API (probably even Chinese one).

thanks for the explanation, I'm still not sure what "weights" are here, but I'm also not sure I want to know anything more about LLMs than I already unfortunately do, so I won't ask you to explain it. :P