you are viewing a single comment's thread
view the rest of the comments
[–] 3 points 3 months ago* (last edited 3 months ago) (2 children)

Full quantisation? I've only got a 8GB 3070, but I'll give it a go

Edit: Tried the unsloth/qwen3.6 with llama.CPP, and it failed to allocate a 26GB Vulcan buffer and died. Dunno what magic your using, no luck for me though :(

  • source
  • parent
  • hideshow 2 child comments