I think it's both, but as I said I didn't read the papers properly. They seem to describe there's more to it. But obvously LLMs are made to regurgutate stuff and they're fed with textbooks and homework assignments and scientific papers. There is some effort to force them not to just memorize stuff. But obviously(?) that's the first thing they do. I don't see why they wouldn't just memorize and regurgitate what they can and simultaneously come up with a way to predict results that are less common in the training dataset. Whether that's proper math or not is a different story.
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments