The wait is over, most ggufs are already up. Nice to see there's models for many different hardware configurations.

you are viewing a single comment's thread
view the rest of the comments
[–] 3 points 8 months ago (3 children)

I just realised this is the much more useful link: https://github.com/p-e-w/heretic?tab=readme-ov-file

I can see at least one -heretic version of a Qwen3.5 model on Huggingface already; can't vouch for quality though.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 3 points 8 months ago (2 children)

    Thanks! I'll wait a few days, maybe one of these pops up on Huggingface. Are "abliterated" versions alright these days? Last time I downloaded something with that word in the name, it wasn't very good.

  • source
  • parent
  • hideshow 2 child comments
  • [–] [S] 4 points 8 months ago* (1 child)

    I don't follow the discussions on this topic very closely, but as I understood, there are different ways to achieve the goal, but all impact quality to some extent. Heretic is discussed as one one of the SOTA methods. The README posted above states the following, so it seems that heretic is some sort of next gen abliteration.

    It combines an advanced implementation of directional ablation, also known as "abliteration" (Arditi et al. 2024, Lai 2025 (1, 2)), with a TPE-based parameter optimizer powered by Optuna.

  • source
  • parent
  • hideshow 1 child comment