Not many people seem to be reading the article vs commenting based on the headline. For those who didn't read it, this has nothing to do with pretraining off the Internet as a data source and instead relates to post-training sandboxing with true air gapping after the Huggingface hack.

To be fair, it's not Lemmy's fault the author of the headline went with a confusing title that misrepresents the actual story in order to seem more sensationalistic. But given how often that happens with stories on the subject, it's worth keeping in mind that only reading headlines and forming opinions off that might lead to a sense of things that are detached from reality.

Was Henry Molaison no longer possessing consciousness after the surgery?

How about Clive Wearing after the herpes caused anterograde amnesia?

There's been a number of humans that due to injury can no longer form new memories or learn anything. According to your definition, do those humans lack consciousness?

Claude Mythos and Fable are the exact same model, the branding only reflects which one has more restrictive classifiers in front of it during deployment. But the weights are identical.

2 only works if regulatory capture in place to prevent cheaper competition from open models.

Ironically there was notable news in the fusion reactor space this week as Gemini 4, building on the work of Matt Landreman and GPT-6 Astra a little while ago, expanded counterexamples to Grad's conjecture which had claimed exact solutions for controlling a key aspect of fusion were impossible.

So fusion may actually be more around the corner thanks to the advances of capabilities of AI which helped prove that the fusion problems aren't as hard as they were thought to be for decades.

Fuck religious extremism and orthodoxy wherever it rears its monstrous head.

There is (it's the kv cache), but it's recomputable from the session transcript.

Part of why comparing human consciousness and possible AI equivalents is not easily apples to apples.

I can't take a human and create a backup state to restore to, or fork into multiple versions of the same mind that then goes down different paths, or pause and then resume things.

Humans do have consciousness continuity breaks and it's quite possible we only have an illusion of continuity vs actual continuity at a neurological level. So just as a transformer has seams at generating the individual token as well as at a context window, humans have consciousness seams (sleep, anesthesia, highway hypnosis, short term memory capacity, etc). But there are very wide differences between human consciousness as an effectively constrained singleton and AI as able to be quite multiple and resumable from an initial single state.

is reversible so completely irrelevant

So is a session transcript.

Self-supervised learning is the key training step whereby the relationships between concepts are formed in pretraining.

You might also benefit from reading over this summary from the authors of the widely replicated and expanded upon 2023 paper finding that self-supervised learning in a toy transformer that was only trained on board game moves in notations like 'a4' developed a model of the full board and tracked board positions for their own and opponent moves.

This ability to form a world model at varying fidelities and accuracy depending on the relevance to the training data was a big surprise to most experts in the field at the time, though now more common knowledge.

Is killing a pig murder?

What about putting someone under general anesthesia?

(It was a nice try at a reductio ad absurrum fallacy, but those tend to kind of suck, just fyi.)

Also, personally I'd say that given the general poor standing of consciousness research in humans across both neurological and philosophical domains, the much more important and relevant question than whether models are conscious (a fool's errand to debate) is whether or not they are simulating consciousness. Lots of reasons engaging with the "as if" conscious makes sense for the latter even if the former is false.

You might want to look into AI training more. It literally does have a '???' step to it, which is why whenever someone talks about how it was 'programmed' or is "just if/then or matrix multiplication" they are telling on themselves as not knowing what they are talking about.

It was not planned. Nearly every single expert in the field a decade ago was on the record saying the capabilities that exist today wouldn't happen.

Scaling took pretty much everyone by surprise, even its proponents.

The problem is that in this case it literally can stop working because you yelled at it.

Power drills aren't compressing thousands of years of human history into hyperdense clusters of memetics that they are then extending in order to function.

If they were, they'd likely come with warnings not to yell at them.