Agreed. Hate the data centers and corporate obsession with extreme waste for total value capture. Not the tech, especially on a local scale.

A 128GB amd strix halo runs at 120W and can host large LLMs at home. If you ran it full time for inference it would still be under $20/mo in electricity costs.

I'd say you should read the article a little more closely, but it's not written very well. But it brings up interesting things that have nothing to do with your local GPU usage. For example, it names an interesting point about simply delivering 4 gigabytes of data to that many people. If pushed out to ~15% of Chrome users without consent:

• That'd be 500 million people
• It would be 2 exabytes of data
• 120 GWh of energy, equivalent to the annual electricity consumption of about 36,000 average UK households
• 30,000 tonnes CO2 emitted, roughly the annual emissions of 6,500 cars

And that's just for the initial data push. Models need ✨updates!✨