cross-posted from: https://sh.itjust.works/post/61139432

I seriously can't believe how much progress he's made for the FOSS community. He actually might take a bite out of the big 3's profits with this

you are viewing a single comment's thread
view the rest of the comments
[–] 4 points 5 months ago (3 children)

I have a rx5600xt (6gb), 32gb ram, ryzen 3600. System hasn't been updated since i built it during covid. QwenV3-vl35B is the heftiest thing I can run, it gets around 2 tokens/sec, in LM studio. It's easier than most people seem to think.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 2 points 5 months ago (2 children)

    How do you now run out of RAM? Does it offload to system RAM?

  • source
  • parent
  • hideshow 2 child comments