Yes, it's different this time:

In the past, people have often used tools from calculators to GPS systems for a kind of task-specific “cognitive offloading,” strategically delegating some jobs to reliable automated algorithms while using their own internal reasoning to oversee and evaluate the results. But the researchers argue that AI systems have given rise to a categorically different form of “cognitive surrender” in which users provide “minimal internal engagement” and accept an AI’s reasoning wholesale without oversight or verification. This “uncritical abdication of reasoning itself” is particularly common when an LLM’s output is “delivered fluently, confidently, or with minimal friction,” they point out.

Many (most?) people have always relied on somebody or something else to do the thinking for them.

AI can just be added to the long, long list of sources people so inclined uncritically accept.

Yada yada here's the open-access paper.

(I usually provide these links neutrally, but I'll make a point here: in a public health community, it may be worth requiring linking to a paper on top of the news article covering it – especially if it's open-access. Ars here is mercifully concerned with methodology; many outlets don't give a shit.)

Conclusion is as follows (for expedience; I encourage reading other parts):

As AI becomes ubiquitous in society, understanding how it reshapes human thought is essential. Tri-System Theory [author's note: introduced in this paper; tenuous to call it a "theory" on that basis] offers a new framework for this cognitive frontier. By introducing System 3 (Artificial) as a distinct and external reasoning process, we move beyond the classical architecture of dual-process theories and chart a new decision-making paradigm: one where intuition, deliberation, and artificial cognition coexist, compete, or converge. We show that people not only use System 3 to assist with reasoning, but often surrender to its outputs whether correct or flawed. This cognitive surrender illustrates the value and integration of System 3, but also highlights the vulnerability of System 3 usage. Similar to how System 1-driven heuristics lead to systematic biases, System 3 has differential cognitive shortcomings that will challenge decision-makers and society at large.

Tri-System Theory is not a warning about AI’s dangers but a recognition of System 3’s psychological presence. We do not merely use AI; we think with it. [author's note] In doing so, we must ask new questions: What happens when our judgments are shaped by minds not our own? What becomes of intuition and effort when a generative, artificial partner stands ready to answer? How do we preserve agency, reflection, and autonomy in a world where users engage in cognitive surrender? We offer Tri-System Theory as a conceptual foundation for understanding these challenges. It is a theory for an age of human-AI algorithmic cognition, and for the decision-makers, researchers, and designers shaping that future

If you're willing to abandon your thinking to AI, I'm guessing you weren't too attached to it in the first place.

If we let people freely continue in this manner, we're going to evolve into two separate species, one of which we might as well call the Eloi, completely unable to think or perform any tasks for themselves.

"ChatGPT, how should I feel about this?"

I wonder if being religious is a factor in which group you belong in.

Religion has been doing this for centuries.

I’m seeing this, even in intelligent people. They expect they can just keep prompting and reach a 100% correct answer that needs no human verification. Looks like an earlier phase of AI Psychosis to me.

The whole point of ai is to train people out of critical thinking. It started with shitting all over / underfunding the arts, then turning schools into employee training camps and now, to remove any last residue of free thought, ai to fill the gaps.

I dunno, I find this entirely unsurprising. And I bet this also correlates strongly with political identity too: authoritarians love gullible idiots that vote for them. Humanity is fucking stupid in aggregate

Wouldn't they... have to have some logical thinking in the first place... ?

I think a very good amount of AI users are like this....

In general, “fluent, confident outputs [are treated] as epistemically authoritative, lowering the threshold for scrutiny and attenuating the meta-cognitive signals that would ordinarily route a response to deliberation,” they write.

People have always conflated confidence with ability and knowledge. That's why so many positions of power are occupied by confident bullshitters. It seems like that tendency transfers over to people's interactions with LLMs.

It would be interesting to experiment with an LLM trained to sound less confident and more tentative or self-deprecatory. Maybe the results would be different.

"those who treat AI as a powerful but sometimes faulty service that needs careful human oversight and review to detect reasoning or factual flaws in responses" Yep.

Reddit comments are a shining example of this.

Agentic✨

Oh, so we've amalgamated all the Facebook conspiracy theories with 4chan conspiracy theories, along with whatever % garbage political messagin,g everyone's major religious texts, and basically the sum off all art, knowledge, and advertising that was available on the Internet at the time.

Of I were real honest, I'd say that last one is the one that really bothers me. The vast majority of our modern media is dumb ads. Really, since the Victorian era. My unscientific guess is that the bulk of modern media is designed to weedle past your logic to make your emotions want to buy various petroleum products.

And out off all that mess we're expecting what?

So like with any other tech or tool, there are two groups of people... Tale old as the world.

This makes clearer that critical-thinking can't be a mere add-on, it has to be the bedrock of people's capability, ideally of 4/5ths of the population as a minimum-standard, XOR social-highjacking by ideologies & AI is certain.

Unless someone's identity is including critical-thinking, then they're prone to cognitive abdication, either to ideology or to LLM, both are the same fundamental abdication.

I didn't know that LLMs were gaining the same abdication that ideologies have been gaining, but .. now that that paper mentions some evidence, it looks clear.

( "religions" are another highjacker of minds.. all religions which displace critical-thinking do the same thing.

For anybody who claims that religion always displaces critical-thinking, you've obviously no experience with Vajrayana style ruthlessly-correct reasoning.

I say Vajrayana style, but it could well be pervasive throughout the different south Asian branches of AwakeSoulism/Buddhism: I don't know.

Western philosophy is mushy-as-hell, by Vajrayana's standards for objectivity & correctness )

_ /\ _

I mentioned that I don’t use AI today, and the person I was talking to was really surprised. They didn’t understand how I could not use AI for anything.

A lot of people never adopted logical thinking in the first place.

Not surprising. We always want to take the path of least resistance. I mean 20 years ago, you could access the world's information via the internet, but you had to know how to search for things. We slowly went from that to "I'll just google it." and "Well, google says." Now that we have LLM's (which IMHO are mostly just fancier, faster google search) we have people legitimately saying "let me ask ChatGPT" and "ChatGPT says."

I think there is a positive future for LLM's and true AI, but its not right now, and it definitely is not in the hands of capitalists.

But LLM's are bad at math and logic.

duh

Yeah, what a new and scary thing that started with AI and nothing else ever. Jfc, AI has been the best thing for liberal moralistic arguments of social degeneracy since social media.

This works well with my AI-as-shackles theory. I’m assuming the rich will use AI to enslave us, and this article makes that scenario seem much easier for them to achieve.

Humanity could actually have reached it's pinnacle in the recent years. Now it's just gonna go downside, with people unable to accomplish to most basic logical operations without ai.

There’s a huge amount of dolts who think AI magically knows the answers to everything and is always right.