Was unable to get GLM 4.5 UD in that quant through LM studio, I'll try just llama.cpp instead

edit: Runs fine in llama.cpp, 5.1-5.6 tok/s on CPU, but I can't seem to fit the whole memory on GPU. Still experimenting.

llama-cli --ngl 93 --context 12288 --no-mmap -t 16

12k context seems like the largest I can get (uses 118GB). You can probably push it further without a GUI, but on a desktop environment gnome daemon starts killing processes.

Prompt processing at 12.3t/s, inference at 10.7-11.1 t/s.

I would say this verges on not-usable between the speed and context window. After the thinking tokens are through you've burned a lot of your usable context.

edit: implementing conway's game of life in numpy worked, used 3k/12k context, and took 7minutes.