The tokenization is a low-level implementation detail, it shouldn't affect an LLM's ability to do basic reasoning. We don't do arithmetic by counting how many neurons we can feel firing in our brain, we have higher level concepts of numbers, and LLMs are supposed to have something similar. Plus, in the """thinking""" models, you'll see them break up words into individual letters or even write them out in a numbered list, which should break the tokens up into individual letters as well.
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments