And books just contain a series of words. You are playing semantics games. An ebook is just a bunch of electrons shoved into a rock. Yet, somehow, it also contains knowledge.

In a less abstract way, LLMs giant relational databases that are expressed by using the relationship between previous words to select a word that might follow. It's in the relational graph that the knowledge is stored. The information is there, and the way we've figured out best to get it out is by generating tokens.