There are rumors about a new AI Chat coming soon based on Llama 3. Is that true? Don't get me wrong, the current AI Chat is great, but its goldfish memory ruins any chance to roleplay with an elaborate lore or world-building, not to mention the constant repetition of expressions like "maybe, just maybe" and so on.

you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 1 year ago

Context tokens are not directly based on the model unless talking about very large token counts. Context tokens come from the amount of memory it is given to work with, which costs more to host. Some models (not any of the Llama models) are better at managing this memory, but it still uses a lot of resources to host a large token context.

  • source
  • parent