you are viewing a single comment's thread
view the rest of the comments
[–] 5 points 2 weeks ago (4 children)

At 3 tokens a second and calling it useble I would guess?

I see people claiming that shit all the time and it's baffling.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 2 points 2 weeks ago (2 children)

    They said 50- i looked up strata, and it uses smaller quantizations/optimizations of that model depending on your system.

    I have 48g vram and 192g system ram and get ~15t/s on qwen 3.8 flash next at Q4. Off the top of my head. I think Q6 is 12.

  • source
  • parent
  • hideshow 2 child comments