you are viewing a single comment's thread
view the rest of the comments
[–] 4 points 6 months ago (3 children)

Can you explain how it's more than probability? It's using a neural network to guess the most likely next token, isn't it?

  • source
  • parent
  • hideshow 3 child comments
  • [–] -1 points 6 months ago (2 children)

    The fact that it uses a non-trivial neural network. If it was simply a rate count of based on a corpus of how much time each word is followed by each it wouldn't be stronger than keyboard word predictions. To make accurate suggestions requires emergence of primitive reasoning on the semantics of the tokens, LLM neural networks (transformers) can be analyzed to find subnetworks dedicated to modeling reality. It is still probability, but saying it's just probability is not faithful

  • source
  • parent
  • hideshow 2 child comments
  • [–] 4 points 6 months ago (1 child)

    It's still just predicting the next token, it's just using more past data points than your keyboard. The rest of the phenomena are emergent from that. I think it's important to keep that in mind given how much they can imitate human reasoning.

  • source
  • parent
  • hideshow 1 child comment