I felt sorry for it and stopped it, 14min and the poor thing was just going around in circles.

I'm just looking at running some local llm on my m2 16GB mac and this was with the default settings for the qwen3.5 9b in LM Studio. Any pointers to get the best out of a M2?

you are viewing a single comment's thread
view the rest of the comments
[–] 7 points 2 weeks ago (7 children)

Try Ternary Bonsai. It's a heavily quantized version of Qwen 3.8 27b. It won't do wonders, but it shoud be better: https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf-dev The page says you need a fork of llama.cpp, but the support was included in the latest release (0.7). You can read more about the mode on the main model page: https://huggingface.co/prism-ml/Ternary-Bonsai-27B-gguf

  • source
  • hideshow 7 child comments
  • [–] 3 points 2 weeks ago (2 children)

    Oh god no. Bonsai is such a waste of time! Totally a dull version of Qwen. Keeps getting basic things wrong and isn’t agentic at all.

    But I’d be curious to hear if you’ve used it and had a different experience.

  • source
  • parent
  • hideshow 2 child comments
  • [–] [S] 1 point 2 weeks ago* (last edited 2 weeks ago) (2 children)

    I had already downloaded a Bonsai 27B, just not really tested it yet

  • source
  • parent
  • hideshow 2 child comments