On the bar chart I included what I think are the most popular models right now (not objectively, based on my anecdotal observations, link to include/exclude what you want).

What do you think? Is it fair? There was, and to some digree still is, a bit of controversy about AA score indexes after they updated their benchmarks to V5.0. I am mainly interested in your thoughts about QFN in relation to other models.

you are viewing a single comment's thread
view the rest of the comments
[–] 4 points 17 hours ago (3 children)

I'm running Gufo on a Strix Halo PC with 128GB shared memory.

The quant is unsloth's UD-Q4_K_XL. Speed is really good with this configuration (1600 t/s prefill and 100 t/s decode). I wanted to try some higher quants as well but I'm waiting for Qwen 4 for now to do more testing.

You should be able to get QFN running quite well on your system as well when offloading unused experts to RAM.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 3 points 15 hours ago (2 children)

    Amazing, will give that a shot. Thanks!

  • source
  • parent
  • hideshow 2 child comments