you are viewing a single comment's thread
view the rest of the comments
[โ€“] 9 points 5 months ago

That's the biggest baddest model out there. There are models that get you 90% of the way there with a significantly smaller parameter count and thanks to MoE offloading you don't need the entire model active at once.

A 5090 and a beefy CPU and tons of RAM won't be cheap or even affordable to most, but you could run very big models and have a beefy PC for other activities. But even a 16 GB card could do plenty.

  • source
  • parent