On the bar chart I included what I think are the most popular models right now (not objectively, based on my anecdotal observations, link to include/exclude what you want).

What do you think? Is it fair? There was, and to some digree still is, a bit of controversy about AA score indexes after they updated their benchmarks to V5.0. I am mainly interested in your thoughts about QFN in relation to other models.

you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 6 hours ago* (3 children)

What sort of setup are you rocking? Which quant? I'm running Qwen 3.6 35B A3B Q2 for speed on my 16GB T4060 Ti and Qwen 3.8 27b Q5 on my R9700 32GB for coding. Unsure if I can run a half decent QFN quant on my setup along with my 128GB of sys RAM.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 3 points 5 hours ago (2 children)

    I'm running Gufo on a Strix Halo PC with 128GB shared memory.

    The quant is unsloth's UD-Q4_K_XL. Speed is really good with this configuration (1600 t/s prefill and 100 t/s decode). I wanted to try some higher quants as well but I'm waiting for Qwen 4 for now to do more testing.

    You should be able to get QFN running quite well on your system as well when offloading unused experts to RAM.

  • source
  • parent
  • hideshow 2 child comments