you are viewing a single comment's thread
view the rest of the comments
[–] [S] 18 points 3 months ago (2 children)

I mean baking knowledge into a model isn't really all that useful to begin with. Just download wikipedia locally and have it access it through tool use, it's way more efficient and more accurate. And yeah, I find Q6 tends to be the sweet spot where it's close enough to full 16 bit in performance, but doesn't chew up too much memory.

  • source
  • parent
  • hideshow 2 child comments