you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 11 months ago*

That's not strictly true.

I have a Ryzen 7800 gaming destkop, RTX 3090, and 128GB DDR5. Nothing that unreasonable. And I can run the full GLM 4.6 with quite acceptable token divergence compared to the unquantized model, see: https://huggingface.co/Downtown-Case/GLM-4.6-128GB-RAM-IK-GGUF

If I had a EPYC/Threadripper homelab, I could run Deepseek the same way.

  • source
  • parent