The change is a result of MTP support landing in llama.cpp. The Qwen3.6 Unsloth GGUFs are now out of experimental mode, with llama.cpp has merged many PRs, and MTP is now properly supported in Unsloth.

https://unsloth.ai/docs/models/qwen3.6#mtp-guide

you are viewing a single comment's thread
view the rest of the comments
[–] [S] 2 points 6 months ago (2 children)

@davel@lemmy.ml the requirements for running Qwen just got significantly lower, it's basically the best local model at the moment

  • source
  • hideshow 2 child comments
  • [–] 1 point 6 months ago (1 child)

    Thanks. I haven’t bought hardware to run things locally yet. I did buy some DeepSeek tokens this weekend to play around with. Maybe I should rent until the bubble pops and then buy a supercomputer at fire sale prices.

  • source
  • parent
  • hideshow 1 child comment
  • [–] [S] 2 points 6 months ago

    Oh yeah, that's definitely the best approach if you don't already have the hardware since DeepSeek is just absurdly cheap to use. Eventually, hardware prices are going to come down, and local models are going to keep getting more efficient too. So, dumping a few grand on a rig right now doesn't really make much sense.

  • source
  • parent