255
People who use AI are actual NPCs (lemmy.world)
submitted 1 week ago by to c/showerthoughts@lemmy.world
168 comments fedilink hide all child comments
 

All their responses are generated, no conscience there, just AI. "Another brick in the wall" hits stronger than ever now.

you are viewing a single comment's thread
view the rest of the comments
[–] 5 points 1 week ago (13 children)

I am pretty sure if you asked people with knowledge if they want a society where accumulating knowledge is no longer an exceptional trait, they would have also said “no”. When I was a kid we were still told using calculators is bad; yet having calculators everywhere meant we could suddenly solve bigger issues at a larger scale.

Lots of technological progress has brought significant change. And no doubt all of them also brought big issues with them. But ultimately all of them helped us advance further.

I also don’t like that my (work) life changes considerably due to LLMs. But that cat is out of the bag. Now I can only try to adjust accordingly or I will simply be left behind.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 13 child comments
    •
  • [–] 1 point 6 days ago* (last edited 6 days ago) (9 children)

    Because you seem to be giving measured responses, even though I and other vehemently disagree with you, here's where I think you're missing things:

    computers, the internet, and calculators make many tasks significantly easier. That's the point of technology. However, if you don't understand the how and why of math, you'll have no idea how to use a calculator. If you just take whatever the first result of an internet search tells you at face value, you're demonstrating a complete lack of critical thought. You have to be able to analyse sources, or you'll just be mired in mis- and disinformation. If the only thing you know how to do on a smartphone is search the internet or download and use an app, then not only are you inefficiently using the technology, but you are also making yourself incredibly vulnerable to security and privacy threats.

    The same is true for LLMs, but doubly so, because everything they output is in natural language (or code output, in the case of coding). If you are an expert coder, then it is possible to use LLMs to significantly increase your efficiency. If not, though, you completely offload the entire problem to the LLM, and can't even check if the solution is valid because, like the internet, you have to have the knowledge necessary to question the source. If you make the assumption that it's responding to you as a person would, then you're screwed, because you're incorrectly ascribing knowledge to a stochastic parrot. LLMs have all the worst cognitive pitfalls of the smartphone, the calculator, and the internet, but only actually have the strengths of the internet, because they're only as good as the training data. Every single thing an LLM outputs must be treated with scepticism because the issues aren't in every output, but could be in any output.

    In a world filled with LLMs being used as expert systems, the low-skill users will, inevitably, fail to actually gain skill from using the tool, because the flaw in the tool is inherent (it isn't actually an expert), and you require preexisting knowledge to be able to check its output. In exactly the same way that an LLM cannot reliably check its own output for errors, low-skill users are similarly unable to do so, and will simply offload their own cognition. This leads to a decreasing number of experts who are actually able to use the system efficiently, and an ever-growing layer of untermenschen who must simply trust the mechanical parrot like some oracle, with about the same level of accuracy as your average cold-reading psychic.

    This is not an issue of those with knowledge trying to privilege their own skills. It is that experts are warning that this abomination, which the public wants to use like an expert system, is not and can never be an "expert". Non-experts simply think it's entrenched power trying to protect itself, but it's not. The data agrees with my points above. I can show you several particularly damning studies which demonstrate my point.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 9 child comments
    •
  • [–] 5 points 6 days ago (7 children)

    I can show you several particularly damning studies which demonstrate my point.

    Well, let's have them then.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 7 child comments
    •
  • [–] 0 points 6 days ago* (last edited 5 days ago) (6 children)

    Gladly, though tonight I have to write 5 more letters of recommendation. Have one for now, and I'll edit this when I have time. (I have edited in some more). Hopefully this one, the most damning in my opinion, shows how the vast majority (80%) of low-skill users will inevitably choose to become so cognitively reliant on AI that their performance will not only fail to improve, but actively deteriorate, making it impossible to become the sort of expert user necessary to actually use LLMs without the pitfall.

    The Generative AI Learning Penalty: Evidence from Chinese Secondary Education

    Hopefully that tides you over until I have more time to find the other studies I read!

    Edit: here's another which is more qualitative, but no less damning:

    AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking

    I also can't help but point out my second-favourite AI study, Vending Bench, which shows how an LLM will, for no apparent reason, decide to threaten nuclear annihaltion over an hallucinated error in shipping. LLMs are not expert systems. They cannot be experts. LLMs cannot consistently operate without constant expert oversight, even for something so simple as stocking a fucking vending machine.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 3 points 6 days ago (5 children)

    I think my problem with things like this is I heard the same bullshit growing up about using calculators or the internet, or cell phones or then smart phones.

    Having access to any info you need is bad actually because now you don't just have to remember. Solving problems with calculators is bad because you don't have to actually do the hard work of the "real math".

    Just because people can choose to be idiots doesn't mean everyone will, and people chose to be idiots long before LLMs gave them access to potentially being correct more often.

    Idk man, Lemmy's hate boner for AI goes far beyond reasonable levels into irrationality, its almost ironic if it wasn't so fucking annoying. You can't have a rational discussion, just look at this thread. Literally "you're a fucking npc if you use AI."

    I grew up in interesting times for computing, and I already understood at a young age how little most people using them then even actually understood how they work because of the abstraction, then smart phones even moreso. So from that aspect I get the concern, however I do not think we can force people to try and understand by shaming them for not being fucking nerds like us.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 5 child comments
    •
  • [–] 1 point 6 days ago* (last edited 6 days ago) (4 children)

    Did you read either my original point or the study you're replying to? My point is that the problem is with the nature of the tool first, and with the fact that people seem to treat it like some fucking oracle second.. The fact that people misuse it is not some referendum against the populace, but against the fucking psychotic corpocrats and those who fellate shitlords like Altman, because they're pretending that LLMs are the solution to every problem, and as if they're true "AI". That's what needs to be fought, and it can only be done with education. I'm not sure why you're replying to me, when literally every point, including your ad hominem indicating that I have some "hate boner", or the one where you imply that I must agree with the thesis of the post to engage with a comment in it, can be easily seen as farcical by anyone with the literacy of a thirteen-year-old.

    1. in my original comment, my whole point was that these other tools are different, and that LLM use can only be responsible when checked, and that users in the first years of their profession are unlikely to be able to do the hard checking required to turn hallucinated nonsense into something actually useful. The consequence is that, unlike a calculator or a computer, these LLMs will lead to a decrease in experts over time, reducing the number of those who can actually use LLMs efficiently and, more importantly, functioning as a barrier to those attempting to actually become experts. It requires skill to use a calculator at all, and it requires skill to use a computer for any given task. LLMs are more like the internet, which requires no skill to use to generate an answer, but which will constantly produce inaccurate information. The difference is that LLMs are worse, because there's no part of the LLM which can look at that output critically and say "actually, that last commenter was an idiot, and the real solution is this".
    2. In the study to which you were ostensibly replying, tens of thousands of students were given free choice over whether to use AI, and how to use AI. Of those who chose to use AI, 80% chose to offload their cognition. Those who did saw approximately a 22% drop in performance in_every single subject_. Not everyone will choose to be idiots, but even in a cultural context where performance on a single test determines the entire trajectory of your life, over three quartets chose the idiocy. If you can't see how that's a direct counterpoint to everything you just said, I can't help you. You should really read the study, because it's worse than even I summarised here.
  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 2 points 5 days ago (3 children)

    I think you're misunderstanding my point a bit, and I'll admit the calculator comparison was overly simplistic. I'm not saying LLMs are the same as calculators or that they don't have unique problems. I'm saying those problems don't automatically make the technology inherently detrimental to learning.

    And I actually agree with you about people blindly trusting LLMs. I see this shit at work all the time in tech support. People will throw a question into Copilot, get a confidently incorrect answer, and then pass it along to other people without even attempting to verify it. It's incredibly frustrating, especially when they're doing a job where they really should be developing the knowledge to recognize when something is wrong.

    But that's kind of my point. There's a massive difference between using an LLM to avoid thinking and using one to help you think, learn, and accomplish things.

    The Chinese education study is genuinely concerning, I'm not dismissing it. But the 80% figure is based on behavior consistent with outsourcing homework, not the researchers somehow knowing that 80% of students consciously chose to stop thinking. And the students who continued putting comparable effort into their homework experienced much smaller learning losses. That seems like a pretty important distinction.

    The other study shows a correlation between AI use, cognitive offloading, and lower critical thinking scores. Again, worth taking seriously, but that doesn't establish causation or prove that using LLMs prevents someone from developing expertise.

    And Vending-Bench? Yeah, LLMs can do some absolutely stupid shit when left to operate autonomously. I don't disagree. But how does that invalidate using one as a tool under human direction? I'm not asking an LLM to independently run my life or make decisions without oversight.

    I use LLMs constantly for development work. I've been a computer nerd since I was a kid, taught myself Linux, networking, messed around with programming, and all sorts of other things through documentation, forums, and tutorials. But programming was always something I struggled to actually do, despite understanding a lot of the underlying concepts.

    LLMs have helped bridge that gap. I can now build software I literally couldn't have built before. I can investigate unfamiliar systems, ask questions about concepts I don't understand, explore different approaches, and actually put that knowledge to use.

    And no, that doesn't make me an expert. I'm well aware of the gaps in my knowledge, which is why I use things like SonarQube, CodeQL, testing, and experienced developers to help catch things I might miss.

    But I am learning, and I'm accomplishing things I couldn't before. I don't see how that's somehow a negative.

    I also agree that the corporate AI hype is ridiculous, that these companies oversell what their models can do, and that people treating LLMs like oracles is a serious problem. You'll get no argument from me there.

    Where I disagree is jumping from "a lot of people use these tools poorly, and that can harm learning" to "these tools will inevitably reduce the number of experts and prevent people from becoming proficient."

    That's a much broader claim than I think these studies actually support.

    And that's really what frustrates me about the AI discourse on Lemmy. There's plenty of legitimate criticism to be made, but it so often turns into this blanket dismissal of anyone who uses AI, like we're all just mindlessly copying whatever ChatGPT spits out.

    I'm not defending OpenAI or Altman or any of these corporations. I'm defending the idea that these tools can be genuinely useful for people who know their limitations and actually want to learn.

    Those aren't the same thing.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 1 point 5 days ago* (last edited 5 days ago) (2 children)

    Again, why are you mischaracterising what i have said, as if I've said there's no legitimate use for LLMs? i just don't get why you're coming at me with that. I said that it is possible to use an LLM to increase efficiency. It was the very first point I made in my original comment, but you keep lumping me in with some nebulous "general discourse on Lemmy" that has some "hate boner" for everyone who has ever touched an LLM. Why do you do that? It's just weirdly dismissive. What, specifically have I said to warrant this characterisation?

    As far as the Chinese study, I contend that it does show precisely that, when given the option to cognitively offload to LLMs, tens of thousands of perfectly capable and mentally fit individuals chose to do that. It's not like cognitive offloading is any different from cheating on some homework. It's the easy way out, and these people thought it would serve them. It appears, to me, that you seem unwilling to apply a study of tens of thousands of low-skill users to the directly-related general case of low-skill users cognitively offloading and experiencing skill atrophy in the workplace. This skill atrophy inherently cannot lead to expertise. I fail to see a distinction between the choice to outsource homework and the choice to outsource coding or any other task for which a low-skill user might employ an LLM.

    You mention that you already have a significant baseline from which you can attempt to check the hallucinations of an LLM. You are using it in concert with other sources to learn. You are not taking it at face value. You clearly are a member of that 20%, and I'd go so far as to bet that the vast majority of people who've found their way to the fediverse are likely to also fit into that 20%. However, if we take as given the total ubiquity of LLM use in coding (surely you can agree that it is now virtually ubiquitous), then the 80% misuse factor indicates that, for every 5 individuals who would otherwise become coders, about 4 of them are likely to experience significant skill atrophy from AI overdependence. That is not a sustainable rate of attrition for a subject requiring expert mastery.

    Also, because you asked, the Vending Bench bit was there as a cited source for thw claim that LLMs cannot be trusted as expert systems. Nothing more, but that's an important and salient point, given that the vast majority of people I've seen seem to expect the AI to function exactly like an expert system.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 3 days ago (1 child)

    I think you are making a category error.

    For your argument to be coherent, it must be the case that low-skilled individuals would otherwise go on to become domain experts if not for their use of large langue models. While the reality is that these individuals would likely never rise to the level of expertise in any case.

    You and your interlocutor were essentially talking past one another. LLMs have a disproportionately negative effect on low-skilled, low-intelligence, or uncritical individuals. LLMs improve personal and professional development while being extremely useful to highly-skilled, high-intelligence, or critically thinking individuals. These two statements are not mutually exclusive.

    The point that's being danced around is that LLMs multiply the divide between those who blindly consume and those who think criticality for themselves, and this is a problem. The larger question of whether it shifts more of the later group into the former, is yet to be studied.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 2 points 3 days ago* (last edited 3 days ago)

    I agree with your assessment on all counts but this: critical thinking, skill and even intelligence are not traits set in stone, but built up over the course of a life. AI thoroughly hamstrings this process at the lowest level, and so I contend that people will, at an unsustainable level, fail to ever achieve that high skill level so long as AI is privileged.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 4 points 6 days ago

    I agree with that. LLMs shouldn't be used as expert systems. They might be used as part of an expert system; for example for initial (!) classification of input. I also think they are exceptionally good at assembling information … as long as you look at the sources after you got the big picture overview from an LLM to verify the details.

    This is also my main gripe with the current hype: everyone tries to shove them into everything and try to use it across the board. I am somewhat optimistic that this will tone down again the more people and companies get into trouble for the consequences of their recklessness. That will likely depend on the jurisdiction, though. If judges and/or law makers start dismissing liability when an LLM is involved ("it wasn't me; it was the AI"), we are fucked. If they treat people using LLMs as operators, they should (hopefully) also keep the blame where it belongs: on the meat proxy.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 1 point 6 days ago* (last edited 6 days ago) (1 child)
  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 2 points 6 days ago

    Locally running LLMs are already good for small tasks. Big tasks still require insane amounts of RAM. I am also pretty sure that OpenAI and Anthropic are doomed. They had to invest ungodly amounts of resources into R&D. But their results are out there. Other players can start from zero with no financial baggage; and they already did. There are many open weight models out there that aren't far from the quality of the frontier models.

    When OpenAI and Anthropic implode, there will still be other companies that can and will continue to offer inference.

    The creepier part is how Nvidia, Intel, etc. are entangled in this whole mess. Because them imploding will not just bring AI down, but will massively affect IT — especially our consumer centered IT — in general.

    The AI bubble bursting won't eredicate AI.

  • permalink
  • fedilink
  • source
  • parent