71
Trusting your own judgement on 'AI' is a huge risk (www.baldurbjarnason.com)
submitted 1 year ago by to c/programming@programming.dev
45 comments fedilink hide all child comments
 

OC below by @HaraldvonBlauzahn@feddit.org

What called my attention is that assessments of AI are becoming polarized and somewhat a matter of belief.

Some people firmly believe LLMs are helpful. But programming is a logical task and LLMs can't think - only generate statistically plausible patterns.

The author of the article explains that this creates the same psychological hazards like astrology or tarot cards, psychological traps that have been exploited by psychics for centuries - and even very intelligent people can fall prey to these.

Finally what should cause alarm is that on top that LLMs can't think, but people behave as if they do, there is no objective scientifically sound examination whether AI models can create any working software faster. Given that there are multi-billion dollar investments, and there was more than enough time to carry through controlled experiments, this should raise loud alarm bells.

you are viewing a single comment's thread
view the rest of the comments
[–] 0 points 1 year ago* (last edited 1 year ago) (2 children)

I’m saying that’s fine, but it should be able to reason that it doesn’t know the answer, and say that.

That is of course a big problem. They try to guess too much stuff, but it's also why it kinda works. Symbolics AI have the opposite problem, they are rarely useful, because they can't guess stuff, they are rooted in hard logic, and cannot come up with a reasonable guess.
Now humans also try to guess stuff and sometimes get it wrong, it's required in order to produce results from our thinking and not be stuck in a state where we don't have enough data to do anything, like a symbolic AI.

Now, this is becoming a spectrum, humans are somewhere in the middle of LLMs and symbolics AI.
LLMs are not completely unable to say what they know and doesnt know, they are just extremely bad at it from our POV.

The probleme with "does it think" is that it doesn't give any quantity or quality.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 1 year ago (1 child)

    Is the argument that LLMs are thinking because they make guesses when they don't know things combined with no provided quantity or quality to describe thinking?

    If so, I would suggest that the word "guessing" is doing a lot of heavy lifting here. The real question would be "is statistics guessing"? I would say guessing and statistics are not the same thing, and Oxford would agree. An LLM just grabs tokens based on training data on what word or token most likely comes next, it will just be using what the statistically most likely next token or word is. I don't think grabbing the highest likely next token counts as guessing. That feels very algorithmic and statistical to me. It is also possible I'm missing the argument still.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 2 points 1 year ago

    Is the argument that LLMs are thinking because they make guesses

    No, it's that you can't root the argument that they don't think over the fact they make stuff up, because humans too. You could root it in the amount of things it guess wrong, but it's extremely hard to measure.
    Again, I'm not claiming that they think, but that we don't know until one or the other is proven.
    Right now, thinking one, or the other is true, is belief.

  • permalink
  • fedilink
  • source
  • parent