874
The Pollution Being Churned Out by AI Data Centers Is So Severe That It's Almost Incomprehensible (futurism.com)
submitted 2 months ago by to c/technology@lemmy.world
159 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 2 months ago (2 children)

Woth thé Qwen 3.6 3.8 something nice happened most US company LLM got beat by miles. NVIDIA nemotron 27b isn't as good as qwen two version prior than the one just released. US are beaten on Local LLMs already.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 months ago (1 child)

    if you start excluding the 1000+ B param models ..
    using smaller models, would initially ease hardware demand by 60% .
    OFC you cant.. and probably shouldnt, ignore and disrespect SOTA flagship models

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 2 points 2 months ago*

    Even "big" open source models like DSV4 and Ling/Ring are very efficient. They're big, but (seemingly) sparser than US models, so they're cheap.

    They run surprisingly well with hybrid CPU+GPU inference on desktops. And thats not even getting into the efficient attention mechanisms.

    I can run DSV4 Flash, barely quantized, with ~1M context on my Ryzen desktop at ~11 tokens/s. If you told me that two years ago, I would not have believed you.

  • permalink
  • fedilink
  • source
  • parent