you are viewing a single comment's thread
view the rest of the comments
[–] [S] 5 points 1 week ago (1 child)

I heard some people use adaptive kv streaming to fit large contexts. Unfortunately, it's not merged into the upstream yet.

  • source
  • parent
  • hideshow 1 child comment