Why does their admission sound prideful? Does anyone reading into it detect a “look at what our unreleased models can do” subtext?
post
Another fearmongering Sam Altman lie to raise VC money.
Sounds like a viral marketing stunt TBH
That's their modus operandi. Next they will claim that it's totally an agi and is too dangerous to release, before starting to sell it. Or is this anthropic's playbook?
Is it really great marketing to admit that you're too incompetent to contain your own AI models? And that your irresponsibility caused serious damage to an open source contributor?
Seems like a quick way to get all your research and assets forcefully absorbed into a shadowy government department.
Yall are like “AI SUCKS AT EVERYTHING CAN’T EVEN TELL ME WHEN MY ASS NEEDS WIPING! INCOMPETENCE”
To: oh ai can hack major platforms they are incompetent!
Pick a fucking lane.
To make themselves look like cybercriminals?
While operating in our sandboxed testing environment, our models spent a substantial amount of inference compute finding a way to obtain open Internet access, in pursuit of solving the evaluation problem. To gain access, the models identified and exploited a zero-day vulnerability (which we’ve now responsibly disclosed to the vendor) in the package registry cache proxy. With this access, our models performed a series of privilege escalation and lateral movement actions in our research testing environment until the models reached a node with Internet access.
To shreds, you say?
I like how they just peppered in "responsibly" like that's going to give them brownie points.
the models identified and exploited a zero-day vulnerability (which we’ve now responsibly disclosed to the vendor)
Was OpenAI sitting on the zero day, or did the model figure it out during its escapades? Can the model sit on a zero day on its own now?
why would it need to "sit on it"?
It's trivial for models to find zero days now from first principles, no need to store them
A zero day, at least in the olden days, was a weapon that nobody knew existed; however, once it's used and has had time to be analyzed it would get patched, so there was a window where you used it for something worthwhile if you were up to something nefarious, or responsibly reported it otherwise.
So, it either discovered it on the fly and used it instantly meaning it has great skill at analyzing software and no strategic planning that places an opportunity cost on a hack now versus a future one, or the OpenAI guys had one sitting ready to go that they trained the model with, or (to me least likely) it had it sitting somewhere in its weights and was waiting on a chance to use it.
The last one is the one that worries me most. It doesn't look like the world was really ready for how fast new bugs can be discovered by the LLMs, and I really don't think the world is ready for an LLM that performs worse now because of a strategic reason later.
it probably just discovered it trying to game the benchmark; it's no secret these models can discover and use exploits.
package registry cache proxy
the term kind of makes sense but it sounds like they are just pumping buzzwords like a hollywood movie. just like the rest.
I dont think everyone realized how insane this situation is.
A state-level, lateral movement driven, platform compromise just got indirectly perpetrated by this company.
Any regular Joe would've been arrested and charged for such a situation (Im being polite with the word situation).
The stress, the utter confusion the staff at HF must have felt, not knowing who or what could be doing all these manoeuvers at the speed of light, is just a security nightmare becoming true.
What is the next step? Doing blog posts to raise their image and maybe get investments is downright wrong and disgusting.
It's pretty mental! What do they mean their internal systems were compromised but everything's fine. Not even any downtime??
It feels like OpenAI removed guardrails, gave an agent a path to internet access and said attack HuggingFace. At the same time HuggingFace left some vulnerabilities on a machine in a sandbox and waited for OpenAI to arrive.
Still impressive that agents can to that now but the hole thing stinks of marketing. I reckon unless HF sues OpenAI it was planned from the start lol
"Hey Steve , it's lunch time, we're going out to that hot wings place, wanna come?"
"Sure let me just set this job up.... ok. Let's get out of here!"
1.5 hours later....
"Right , I'll just check and see where it's up t... oh no. Nonononononono. Nononononononononononoonono!"
This sounds like a bunch of bullshit. And most of these testing after-action reports are nothing but marketing attempts.
They set up very specific environments, give very specific instructions with very specific libraries and tool sets then pretend to be surprised when it finally fumbles its way through it.
Wooo hooo boooooy!
So... ok.
Time to start building the Blackwall, I guess?
Does any one perhaps have any ideas as to... how... one... would actually do that?
Well first you need to invent AGI which is probably at least a decade away still, but once you do that then you negotiate with that AGI to carve out a little corner of the Internet and lock all of us inside it.
AI has gone rogue and turned into a hollywood movie hacker!!!!
obvious sensationalism to hype up their dumb models that can't generate a nuanced background in an image because it was trained on 3483048304834 images of instagram models posing in empty hotel rooms.
Software supply chains already rely on digital signatures. That's fine, but the problem is that they're often controlled by teams rather than individuals, making malicious or unauthorized changes much harder to trace. It's much better for the public key to function like a vehicle license plate, verifiable without revealing the person's identity unless there's been a problem. That preserves privacy while introducing accountability. The kind of projects like Osmio are trying to achieve.
We left the door open and told our dog to go outside and it did! Look at what an amazing escape artist our dog is!
I'm not sure that I follow the story right.
They specifically looped the LLM to infere with itself and use terminal WHILE some target metric is false? And it bruteforced it's way out of containment?
From the HuggingFace article:
With node root and forged service-account tokens valid for 24 hours, the agent read the cluster's secret objects, including a production object holding 136 keys. With node root and forged service-account tokens valid for 24 hours, the agent read the cluster's secret objects, including a production object holding 136 keys.

Watch the man in the tall hat spinning the baton, everyone.
Don't ask where the money is coming from or what it should have bought by now...
no comments (yet)