72
Why your local LLM feels dumber than it is (forum.level1techs.com)
submitted 2 months ago by to c/technology@lemmy.world
103 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 2 months ago* (2 children)

You overestimate most people. Here's a study from 2025

https://arxiv.org/html/2211.13087

Among AI models, Blenderbot stood out: in AI-AI conversations, it was judged human 67% of the times - more often than actual human-human conversations. In human-AI conversations, human participants were labeled as humans 68% of the time, and AIs were classified as AI 56% of the time.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 months ago* (1 child)

    This is an interesting study. I wonder why there’s such a disparity between the number of AI ‘participants’ (10) and human participants (1,916). I wonder how adding additional AI models would impact the data.

    Anyway, this is just benchmarking the AI on 6 language specific tasks. They were programmed and set up specifically to perform whatever specific one task they had to do at the time, that is what computers do. The fact that computers are 1% better at recognizing other computers than humans are is irrelevant.

    AI cannot pass the Turing Test, this article is about “6 Turing like tasks” and it can’t even pass those yet, lmao. You know they’re already seeing diminishing returns in further training right?

    Even in this article it says that even if AI could pass a real Turing Test it wouldn’t be a sign of intelligence or cognition.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 2 months ago

    Something I should point out just in case is that 50% success rate is the same as random chance. So when the study says:

    AIs were classified as AI 56% of the time.

    That really means that humans could not tell the difference between AI and human, since their success rate was almost the same as random chance.

    6 language specific tasks. They were programmed and set up specifically to perform whatever specific one task they had to do at the time, that is what computers do.

    Sure, and that task was "imitate humans", and these computers seem remarkably good at it. That's what I'm saying. I don't care that these AI work differently than humans. They can imitate humans so well that humans can't tell the difference.

  • permalink
  • fedilink
  • source
  • parent