you are viewing a single comment's thread
view the rest of the comments
[–] 47 points 10 months ago* (11 children)

It's pretty obvious how this happened.

All the data it has been trained on said "next year is 2026" and "2027 is two years from now" and now that it is 2026 it doesn't actually change the training data. It doesn't know what year it is, it only knows how to regurgitate answers it was already trained on.

  • source
  • hideshow 11 child comments
  • [–] 13 points 10 months ago (2 children)

    nah, training data is not why it answered this (otherwise it would have training data from many different years, way more than of 2025)

  • source
  • parent
  • hideshow 2 child comments
  • [–] 5 points 10 months ago*

    There's data weights for recency, so after a certain point "next year is 2026" will stop being weighted over "next year is 2027"

    It's early in the year, so that threshold wasn't crossed yet.

  • source
  • parent
  • This instance actually seems more like 'context rot', I suspect google is just shoving everything into the context window cuz their engineering team likes to brag about 10m tokens windows, but the reality is that its preeeeettty bad when you throw too much stuff.

    I would expect even very small (4b params or less) models would get this question correct

  • source
  • parent
  • [–] 2 points 10 months ago (4 children)

    This is actually the reason why it will never actually become general AI. Because they're not training it with logic they're training it with gobbledy goop from the internet.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 5 points 10 months ago (3 children)

    It can't understand logic anyway. It can only regurgitate its training material. No amount of training will make an LLM sapient.

  • source
  • parent
  • hideshow 3 child comments
  • [–] -2 points 10 months ago (2 children)
  • [–] 2 points 10 months ago

    Math, physics, the fundamental programming limitations of LLMs in general. If we're ever gonna actually develop an AGI, it'll come about along a completely different pathway than LLMs and algorithmic generative "AI".

  • source
  • parent
  • [–] 1 point 10 months ago

    Based on what LLMs are. They predict token (usually word) probability. They can't think, they can't understand, they can't question things. If you ask one for a seahorse emoji, it has a seizure instead of just telling you that no such emoji exists.

  • source
  • parent
  • load more comments (1 reply)