I know and love the general threadiverse pushback against big tech bullshit, of which AI is a monstrously overgrown appendage, but I do wish it wasn't quite so militant in cases like this post. It's not mindless AI boosting (it's not AI boosting at all -- the author directly states they're ambivalent about AI), it's a genuine expression of opinions that I think are, if nothing else, interesting.

I don't think it deserves to be downvoted. But I'd love to hear people's counterarguments.

I'll have to wait until I'm back home to give a proper reply, but the article reads really poorly.

Absolute drivel.

There are three sections of bullet points, but the third is numbered for some reason.

The third one is the footnotes. That'd be the reason.

Personally I didn't clock the article as slop; having footnotes in the first place is a credit towards that.

I am not anti ai but that article title is so dumb

Conservatives are protecting pedophiles. They are not credible people

MF couldn't even write his own bullshit article of lies. There are three sections of bullet points, but the third is numbered for some reason.

The writing is manipulative, the points are partial to full lies. This whole thing sucks. I can't believe I took time out of my morning to read this drivel.

Neat that a local LLM can do a decent job.

The bottom half of the beak is messed up though, it doesnt seem to understand the design.

Holy shit LLMs have come a long way in generating SVGs. Last time I saw this attempted, you could sorta kinda tell that it was trying to draw the thing. Now it draws the thing...

Thank you for the Carroll paper. I'm actually looking for stuff like that atm.

In the paper, he caveats

Everything we have said presumes from the start that the world is ultimately physical, consisting of some kind of physical stuff obeying physical laws. There is a long tradition of presuming otherwise, and if so, all bets are off. The well-known issue is then how non-physical substances or properties could interact with the physical stuff.

so I'm very unclear how this paper can present a hard anti-materialist barrier, when he makes it clear that the paper presumes physicalism. I've only read half of it so far, will continue ...

Materialism and QM have no conflict. What QM shows, via the KS theorem, is that reality cannot be objective; it must be contextual, arising from the participatory interactions between objects and subjects. Carroll 2021 is a fairly hard metaphysical barrier which prevents spurious anti-materialist claims by fully shifting the burden of proof to claimants; if you genuinely think that there are irreconcilable problems with materialism then you must give a physics experiment which violates the Standard Model.

The reviewers for Hofstadter don't understand the book, which I have on my shelf and highly recommend. The point of Strange Loop is that the caged-bird metaphor, that there is exactly one mind per one brain, is wrong in both directions: sometimes there's more than one mind in a brain, and sometimes a mind is not wholly contained in a single brain. The only reason to skip Strange Loop is if you're a computer scientist or mathematician, in which case you should definitely read GEB first and Strange Loop second.

Referring to a multimodal AI as an LLM isn't technically accurate either, because it's not just an LLM. There's also the vision/audio encoders, diffusion model, and multimodal projectors.

Let me rebuff your points one by one... After I re-read your comment...

(I gave up due to my short context window.)

AGI is a subcategory of AI just like LLMs and diffusion models are. That term has existed since 1997 and was coined by Mark Avrum Gubrud in an article named ‘Nanotechnology and international security’

By advanced artificial general intelligence, I mean AI systems that rival or surpass the human brain in complexity and speed, that can acquire, manipulate and reason with general knowledge, and that are usable in essentially any phase of industrial or military operations where a human intelligence would otherwise be needed. Such systems may be modeled on the human brain, but they do not necessarily have to be, and they do not have to be “conscious” or possess any other competence that is not strictly relevant to their application. What matters is that such systems can be used to replace human brains in tasks ranging from organizing and running a mine or a factory to piloting an airplane, analyzing intelligence data or planning a battle.

AGI is just a type of AI. It's a term coined long after AI that doesn't define new goals and capabilities that weren't already part of AI research, except that they were rather abandoned. If anything, "AGI" was the marketing term to revitalize and focus on those ideals. So there is nothing wrong with the general public to associate "AI" with what you would specifically describe as "AGI".

Furthermore, I don't understand what this public perception has to do with your claim. If they thought of only narrow AI, then would it not make even more sense to call their LLM-based products AI...

You have to start from physics & chemistry

This is the view of scientific materialism. Scientific materialism - a view - comes into irreconcilable (I think) problems even within quantum physics, never mind philosophy. It is also a view that you have to start from the small and build up. This view also has problems.

Thank you for bringing my attention to these books. I notice that the first two reviews on goodreads for "I am a strange loop" go to lengths to disagree with the message. Indeed one reviewer says

I did not find that Hofstadter compellingly demonstrates that this strange loop is the entirety of consciousness

I won't put more weight on a reviewer than an author, but I do find the reviews interesting.

I have not yet looked up Blackmore's book.

I am also not claiming we do not pattern match. I am saying there is very compelling literature that says we do more.

But maybe I'm entirely wrong.

Douglas Hofstadter's "I am a strange loop" is great reading on the subject, as well as Susan Blackmore's "Consciousness: An Introduction".

You have to start from physics & chemistry - our neurons aren't doing "art & nuance". They're just mapping inputs to outputs.

… anyone who falls into this trap is welcome to study the very latest we know about human consciousness.

Isn't this a statement about humans, rather than machines? Moreover, "It's pattern matching not understanding" is essentially the message I got from this leading AI professor at Oxford https://www.youtube.com/watch?v=CyyL0yDhr7I

There’s not a single thing in science pointing to us being anything more than pattern matching machines ourselves …

I'm almost certain this is completely wrong. Iain McGilchrist's work on brain hemispheres points, as I understand it, to a left hemisphere that manipulates the world in ways he compares with modern AI, while the right is capable of the implicit, art, nuance etc. There is nothing in what I've heard of his work that suggests the right hemisphere operates like a machine. Indeed, I think he is very explicit that that is the opposite of the truth.

What is a "cognitive task"? At what point does fitting a straight line to data stop being a computer procedure and become cognitive? Is everything a computer does AI?

yep, that's what I've been saying. the public is uneducated because the people selling AI want them to be. using the technically correct but often misunderstood umbrella term "AI" instead of the more specific term "LLM" is one of the ways that ignorance is maintained

Then the issue is with the uneducated public - not the use of correct terminology. Changing the definition of terms will just create more confusion, not less.

the point is public perception. when the general public hears "AI" they don't think about all the lesser systems that are technically AI, they think of AGI

Yeah, well, at least we humans don't ever hallucinate and just make stuff up, engage in sycophancy, act overly confident, have bias, or suffer from a short context window.

AI is a term used in computer science to refer to any system that's able to perform a task normally requiring human intelligence - like playing chess or generating natural-sounding language. LLMs are AI systems in the actual meaning of that term.

AI isn’t any one thing. It’s an extremely broad term. It simply refers to any system designed to perform a cognitive task that would normally require human intelligence. The chess opponent on an old Atari console is an AI. It’s an intelligent system - but only narrowly so. Narrow AI can have superhuman cognitive abilities, but only within the specific task it was built for, like playing chess.

A large language model like ChatGPT is also a narrow AI. It’s exceptionally good at what it was designed to do: generate natural-sounding language. It often gets things right - not because it knows anything, but because its training data contains a lot of correct information. That accuracy is an emergent byproduct of how it works, not its intended function.

What people expect from it, though, isn’t narrow intelligence - it’s general intelligence: the ability to apply cognitive ability across a wide range of domains, like a human can. That’s something LLMs simply can’t do - at least not yet. Artificial General Intelligence is the end goal for many AI companies, but AGI and LLMs are not the same thing, even though both fall under the umbrella of AI.