My how fast the AI tech bros move to protect corporations but don't move to protect the average coder, artists, writers.

you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 5 days ago (4 children)

Got some references and keywords for research?

  • source
  • parent
  • hideshow 4 child comments
  • [–] 2 points 4 days ago (1 child)

    If you've got at least 8 gigs of VRAM and at least 32 gigs of RAM you can run some pretty beefy Qwen models using Strata, far more than you normally could. Upgrade that to 12 or 16 and 64 gigs respectively and you can run a pretty decent quant of Qwen 3.8 flash next. It's a LOT bigger than the usual 27 and 35 billion models people run locally that themselves require pretty beefy hardware. Strata is doing some magic by deciding which experts stay in VRAM vs which ones are in RAM vs which ones are so rarely used that they can just be read from the SSD as needed (though it's best if you don't have to result to the SSD as a backup option at all)

  • source
  • parent
  • hideshow 1 child comment