I tried some new ones recently (though I have a 24GB GPU). Qwen3.5 9B is pretty impressive for such a small model for agentic stuff like Claude Code. (I can run the Opus distilled model quantized to 6 bit with the full 256k context and no CPU offloading). Gemma4 26B is good if I don't need agentic stuff or a lot of context (it sucks for agentic stuff). You can probably run the smaller versions of these, or with less context.

Definitely give Gemma4 26ba4b a try

It’s MOE so you should be able to get the same offload, and a4b can be plenty fast.

It has decent world knowledge for the size, and from what I can tell is any at small scale coding in common languages like Python.

I’m using was released at the beginning of August 2025 (From an LLM development perspective, it feels like an eternity to me)

I mean yeah, more than 6 months in AI world is an eternity 🤣

the big ones are gemma 4 and qwen 3.5

I'm using Gemma 4 and it works really really well, it's sad to me that I'm using big tech's model but it's just so far ahead of mistral and others that I have no choice

Qwen is really good with thinking turned off, turned on it has a massive overthinking problem, like you say "hi" and it'll think for 3 minutes on how best to reply

Still waiting for Deepseek to come out with v4 at this stage but Gemma 4 is my current sota self hosted model

I've been using gemma4:26b, it's pretty good, although a bit slow even on a 3090,and idk how smaller versions compare