post
Chain of thought is basically garbage.
It works with coding agents because they get an automated hard failure.
The rest of the time it’s just sampling the latent space around a response and should be trimmed out.
That could work with diffusion models but autoregresive models it’s just polluting the context window with the hopes of finding longer tail tokens.
The analogy I use is, it's like a magician pulled a coin from behind a CEO's ear, and their response was "that's incredible! Free money! Let's go into business together!"
Literally no one ever claimed it had reasoning capabilities. It is a trick to produce a string of characters that your brain can make sense of. That's all.
chain-of-thought models
There are no "CoT LLMs", a CoT means externally iterating an LLM. The strength of CoT, resides in its ability to pull up external resources at each iteration, not in dogfooding the LLM its own outputs.
"Researchers" didn't "find out" this now, it was known from day one.
As for who needs to hear it... well, apparently people unable to tell apart an LLM from an AI.
Admittedly, a lot of people have flawed reasoning too.
I get scoffed at every time I call LLMs "glorified auto-correct" so it's nice being validated.
Anyone who actually has a grasp of how Large Language Models work should not be surprised by this, but too many people, even engineers who should really know better, have drunk the Kool-aid.
Didn't read the paper, but the description suggests that the models which were tested were specifically created with a very limited set of training data. I can understand the argument that this might allow researchers to evaluate one property in a simplified environment, namely logic generalization. But it could also be argued that general logic is an emergent property, and limiting the LLM in this way prevents it from reaching that complexity threshold.
I'm not sure if the intent of including the two transformations in the training data was an attempt to provide the LLM with at least one opportunity to generalize between known phenomena before asking it to further generalize.
I went to college for filmmaking. Being a field that revolves around communication, we ended up learning some fundamentals of communication theory. By definition, communication happens when the "listener" percieves a message - whether that message was intended or not. No matter how much work and thought you put into your message, if you shout it into the void, it isn't communication. Likewise, you can say nothing in a room full of people, and still unintentionally communicate just by being seen.
The percieved intelligence of LLMs very much feels like a result of this. There is no intelligence in the machine, we're just percieving communication from a probability machine, and thinking that it's intelligent and trying to communicate with us.
no comments (yet)