The wait is over, most ggufs are already up. Nice to see there's models for many different hardware configurations.

you are viewing a single comment's thread
view the rest of the comments
[–] 6 points 8 months ago (4 children)

Keep an eye on this: https://huggingface.co/heretic-org

I used to use a -heretic abliterated version of gpt-oss-120b, not for any creative reasons but just to reduce the amount of wasted tokens in its thinking, with good results.

(You can turn off thinking mode with the new Qwen models btw - how you do it will depend on how you're hosting it, but basically it's a flag to the chat template. It won't remove the safety guidelines, but it will stop it telling you all about its internal monologue ;).)

  • source
  • parent
  • hideshow 4 child comments
  • [–] 3 points 8 months ago (3 children)

    I just realised this is the much more useful link: https://github.com/p-e-w/heretic?tab=readme-ov-file

    I can see at least one -heretic version of a Qwen3.5 model on Huggingface already; can't vouch for quality though.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 3 points 8 months ago (2 children)

    Thanks! I'll wait a few days, maybe one of these pops up on Huggingface. Are "abliterated" versions alright these days? Last time I downloaded something with that word in the name, it wasn't very good.

  • source
  • parent
  • hideshow 2 child comments
  • [–] [S] 4 points 8 months ago* (1 child)

    I don't follow the discussions on this topic very closely, but as I understood, there are different ways to achieve the goal, but all impact quality to some extent. Heretic is discussed as one one of the SOTA methods. The README posted above states the following, so it seems that heretic is some sort of next gen abliteration.

    It combines an advanced implementation of directional ablation, also known as "abliteration" (Arditi et al. 2024, Lai 2025 (1, 2)), with a TPE-based parameter optimizer powered by Optuna.

  • source
  • parent
  • hideshow 1 child comment