you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 1 month ago

Someone reported q5_1 having indistinguishable results from q8 for kV cache up to 130k IIRC. You could give it a try.

  • source
  • parent