284
I Asked AI to Count My Carbs 27,000 Times. It Couldn’t Give Me the Same Answer Twice. (www.diabettech.com)
submitted 6 months ago by to c/technology@lemmy.world
70 comments fedilink hide all child comments
😂 1
 

Hacker News

you are viewing a single comment's thread
view the rest of the comments
[–] 77 points 6 months ago (10 children)

When are people going to realize that an LLM is not a calculator and doesn't actually know anything?

  • permalink
  • fedilink
  • source
  • hideshow 10 child comments
    •
  • [–] 1 point 6 months ago (7 children)

    That it is not a calculator and is horrible at determinism is not debatable, however its (very biased) huge knowledge is its core feature

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 7 child comments
    •
  • [+] 7 points 6 months ago (4 children)
    [deleted]
  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 4 points 6 months ago

    Bullshit factory is very apt. I was using it for an open book exam and it gave answers entirely skewed to the way the question was asked.

    For example, if I asked “is X bacteria a pathogen in Y disease”, it would say yes, it was a very bad pathogen.

    If I asked “what effects does X bacteria have in this body system”, it said it was a beneficial bacteria.

    Never trust the AI summary, you have to fully read the studies.

  • permalink
  • fedilink
  • source
  • parent
  • [–] -3 points 6 months ago (2 children)

    It does lie and hallucinate a lot, especially with biased context in the question (the bullshit part). The (biased) knowledge is hiding somewhere in its weights, it is just that it is sometimes quite hard to recover.

    Your 40% depends a lot on how you ask the questions and the field of these questions. Humanity's last exam is a morr obiective benchmark for measuring the wide knowledge of LLMs.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 3 points 6 months ago (1 child)

    The models themselves are actually entirely deterministic. The non-determinism you see is actually artificially introduced at the application layer to make the output seem more human. It's usually controlled by a setting called "heat", which when set to 0 will give completely reproducible results.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 3 points 6 months ago

    This is correct, I suppose you're talking about the final softmax layer? When I said they are bad at determinism, I was talking about reasoning on deterministic rules not having deterministic output. For example, LLMs make logical deduction errors, calculation errors etc.

  • permalink
  • fedilink
  • source
  • parent
  • [–] -4 points 6 months ago*

    Probably never. Just like people never realized how computers work, how networks work, how businesses work, how economies of scale work, how financial markets work, how…

    We the people don’t give a shit about how anything works, for the most part. Exceptions include your narrowly focused expertise. We convince ourselves that we understand things, using top-down perspectives, because it’s easier than actually understanding things from a bottom-up perspective.

    Even the strongest critics of AI can’t substantively explain how AI works. They use misnomers like “glorified autocomplete” to reason about it’s inaccuracy, rather than understanding the fundamental limitations of the approach used.

  • permalink
  • fedilink
  • source
  • parent