xtr0n comments on GPT-6 Astra performs unsanctioned supply-chain attacks in simulations … more frequently than previous OpenAI models.

[–] xtr0n@sh.itjust.works 18 points 1 month ago (3 children)

Yeah. It’s a ham handed attempt to gently deflate the bubble instead of letting it pop at some random point in the future. Which, on the one hand, fuck those guys, but on the other hand, at this point, gentle bubble deflation is probably the best possible outcome for everyone.

Also the “it will kill us all” kinda sounds like a drunk at a bar claiming that their hands are registered as lethal weapons. Are nuclear weapons systems somehow internet accessible? Are labs gonna let AI systems control biological research, drone building or nanobot production without human oversite? I guess it could happen but only after some monumentally irresponsible lapses by the people entrusted to safeguard critical systems. Shit. Are we totally fucked?

permalink

fedilink

source

parent

hideshow 3 child comments

[–] Rhaedas@fedia.io 5 points 1 month ago (1 child)

It's not the AI that is the problem. I do mistrust it (and you can lack trust in a system that isn't intelligence, misalignment happens in more than AGI). But it's the humans making stupid decisions for money and glory vs. considering the risks. The humans will make mistakes that will cause things to get out of control, we aren't at a level where the AI is intelligent and out thinking its creators. We just have dumb creators, which is funny because to do that work requires being smart. Maybe just not enough street smart, certainly not in the ones making the decisions.

permalink

fedilink

source

parent

hideshow 1 child comment

[–] xtr0n@sh.itjust.works 3 points 1 month ago* (last edited 1 month ago)

We just have dumb creators, which is funny because to do that work requires being smart. Maybe just not enough street smart, certainly not in the ones making the decisions.

The rank and file workers put all of their INT points into math, computing and problem solving. Most of the folks doing that work had very little time to take the philosophy literature and history courses that would give them context and framework to look beyond their immediate metrics and goals.

The decision makers are kinda victims of their own success. When you’re that high up and have that much power over most of the people in your day to day life, it’s hard to get honest feedback or pushback and it’s easy to disregard what little you actually receive. They believed their own hype and have allowed themselves to be dumbed down by yes men. And really, you can’t get to that level without an incredible string of luck. Hard work and smarts only go so far, you alao need to win an improbable string of dice rolls. I suspect that the human brain has a hard time keeping that kind of outsized luck and reward situation in perspective.