If open weight models are the future, U.S. AI companies are going to have a hard time
(www.fastcompany.com)
You want this one:
https://huggingface.co/turboderp/Qwen3.6-27B-exl3_3.30bpw/tree/main
Or maybe the 3.5bpw one if you don’t mind less context, or 3bpw if you need more:
https://huggingface.co/turboderp/Qwen3.6-27B-exl3
For faster inference at the cost of a little more VRAM usage:
https://huggingface.co/turboderp/Qwen3.6-27B-DFlash-exl3
And you run those in:
https://github.com/theroyallab/tabbyAPI
And FYI, if you have 64GB of RAM or more, you might consider hybrid inference instead.