you are viewing a single comment's thread
view the rest of the comments
[–] 8 points 1 year ago* (3 children)

The OpenAI research identified three mathematical factors that made hallucinations inevitable: epistemic uncertainty when information appeared rarely in training data, model limitations where tasks exceeded current architectures’ representational capacity, and computational intractability where even superintelligent systems could not solve cryptographically hard problems

Perhaps it's because I'm dumb, but does anyone else feel like there are lots of buzzwords here?

Like, does "cryptographically hard" refer to NP-hard or something??

This may be a mean thing to say, but could parts of this article be generated by an LLM?

  • source
  • hideshow 3 child comments
  • [–] 4 points 1 year ago (1 child)

    It all means something.

    The first case is when it has limited samples on a particular topic, leaving it with insufficient data - "epistemically uncertain"

    The second is when it tries to think about something, but it doesn't have the ability to hold it in mind, the thought is in some way too large, that's representational capacity.

    The last is just, factoring stupid large numbers, and things of that nature.

    We're not going to read this one deeply, because we're pretty sure this was demonstrated a while ago. Cool for OAI to catch up with the current state of the field, we guess?

  • source
  • parent
  • hideshow 1 child comment