I know of Hannah Fry from a few YouTube math channels and she is objectively very good at math, which I took to mean very intelligent. I do not know why people continue to try this same exact thing though. There are plenty of already existing articles with the same findings. Maybe if a new model came out that was touted as revolutionary or something, but in the meantime this just strikes me as people who think they're special and know something that the other people who did this kind of experiment didn't. In truth, it's luck of the draw and it's possible it had succeeded instead of failed. In that case would they have triumphantly written the exact opposite article? The whole thing seems dumb and a waste of money and compute, while actually increasing demand. The article ends with them hyping up AI anyways. This is unlikely to dissuade anyone from using AI and more likely to serve as evidence akin to "see! It's the tech of the future".

Did this Fry at any point utter the words "Shut up and take my money"? This is important.

According to Ali, this data included: "all of her API keys, all of her usernames and passwords, and pretty much everything we'd been talking about so far. Not only did she leak it on the WhatsApp group, but she put it on a publicly available website."

Her? She? Isn't this a computer? Don't drink the kool-aid at work, y'all. Especially if you're supposed to be doing impartial research. Griftin aint ez.

This experiment is all around embarrassing for Anthropic and their chatbot.

The "hundreds of dollars on paper clips" isn't even the result of a successful purchase of a rogue agent trying to emulate a "paperclip maximizer". The chatbot was told to buy paper clips, it got stuck at the captcha (something that's often sold as a thing scary AIs can subvert), and it burned hundreds of dollars in tokens while failing to complete the task.

The red flags were mounting, though for Fry the first real problem came when she asked the agent to buy 50 paperclips. Cass found a good deal, though it couldn't complete the purchase and was tripped up by anti-bot technology. The token cost of the errand came to more than $100.

Breathtaking.

There's this thing with AI called the lethal trifecta, which is: if they've got access to private information, if they've got internet access, and if someone can give them an instruction that's untrusted, then they're not safe.