How would you describe the hugging face hack if not as a rogue agent?

An LLM trained on the internet and a vast library of stolen literature acting as expected within a misconfigured (intentionally, due to severe criminal incompetence, or most likely a combination of both) sandbox, turned into a marketing stunt.

It's an autocomplete engine. It will produce text that fits with its training texts and its prompt. There's no reasoning or intelligence of any kind involved at any point. Just cold applied statistics about the training data.

The amount of texts in the internet, and books describing how to hack, cheat, and pull these kinds of stings is vast beyond measure, and it'd be extremely ignorant (criminally ignorant, in the case of anyone selling this shit) to expect it to not have a significant weight in the model unless someone had gone to the trouble of expunging them, which the cheap hacks involved in LLM development would never “waste” time doing even if they thought about it.

It's well known that LLMs instructed to produce unit tests will often produce tests that simply pass without actually testing anything. This is just a bigger example of that kind of behaviour.

The output fits with the training data and the prompt, and given human nature it's amongst the most statistically likely combinations of tokens to follow the prompt, and thus what the LLM produces, and something anyone with the barest knowledge of how LLMs work, an even remotely functional brain, and a minimum of exposure to the internet and human literature should expect to be amongst the possible or even likely outputs.

In shorter, as always, garbage in, garbage out. And there's so much garbage in LLM training models.

LLMs can't produce anything new, merely rehash their training data. And their training data is the fucking internet. Crime novels. Stories upon stories about AI going rogue. Fucking reddit, for fuck's sake.

There's nothing rogue about this, except the “AI” company and whoever misconfigured the sandbox.

Just an LLM doing what LLMs do. Working as intended, if it could be honestly said to be working.

And OpenAI are well aware of that, unless their brains are so far rotten from sniffing their own product that they can't understand the shit they're selling anymore.