TheObviousSolution comments on Sam Altman Absolutely Hates the Idea of People Running Their Own AI Models at Home, Argues They Shouldn’t Own Computers: He'd prefer if we surrender all our RAM to data centers.

[–] Allero@lemmy.today 30 points 6 days ago* (last edited 6 days ago) (17 children)

I mean, he has a point. Hardware is used way more efficiently in the data center, and any sort of deficit is best resolved in a centralized system due to more efficient resource allocation. The same setup could be used like 2 hours a day for your personal needs, or 24 hours a day to serve many people.

THAT SAID, their greed and unsatiable desire to make the number go up by hooking us into subscription economy with ever rising costs is what led us here and it leaves us with little choice but to accept the less efficient way and host these things ourselves.

Remember the times not so long ago when self-hosting was an enthusiast hobby instead of necessity? Barely anyone non-geeky bothered with it, because what existed served people well. Now, having to build our own services remains the only sane choice, and the only way out of the nightmare Sam, among others, has built with his own hands.

permalink

fedilink

source

hideshow 17 child comments

[–] martti@suppo.fi 3 points 6 days ago* (3 children)

I might argue with centralized systems being the only ones that allocate resources most efficiently. You can have nearly same efficiency with decentralized models, except they are more complicated to implement, but the diversification is often worth it.

permalink

fedilink

source

parent

hideshow 3 child comments

[–] TheObviousSolution@lemmy.ca 4 points 6 days ago (2 children)

In another timeline where RAM and GPUs were still largely consumer accessible, I wouldn't doubt there would already be a SETI-like decentralized alternative for LLMs.

permalink

fedilink

source

parent

hideshow 2 child comments

[–] T156@lemmy.world 1 point 6 days ago (1 child)

I don't know about that, mostly because the structure of LLMs at the moment make them incredibly bandwidth-hungry, and also quite sensitive to latency.

You do have some small-scale versions of that, where people link a few smaller computers together to run a larger model, but that usually exists in the form of things like connecting machines via USB4/Thunderbolt, or Oculink, which are much faster and lower latency than a typical internet connection.

There is something a bit like that, with the Kobold AI Horde, but that is still people hosting an entire model and sharing that over the internet, rather than something like Folding/SETI@Home, where you're doing part of the actual calculations yourself.

permalink

fedilink

source

parent

hideshow 1 child comment

[–] TheObviousSolution@lemmy.ca 3 points 6 days ago

I was actually thinking more like participants who can offer computational power during periods of no use, accrue reputation the more they contribute, and can use that reputation to offload computation to the cloud. If your PC was not powerful enough to run local LLMs, it would need to contribute a lot more, or perhaps it would just be limited to clouds of people running powerful enough GPUs for the local LLMs being used so they could get an added benefit to leaving it running at night.

An LLM model running over across multiple PCs across the Internet would not really be workable, no, but their product is.