Yeah.

It’s not even about efficiency, really, but independence from corporations, privacy, and principle. Kind of like Lemmy.

People will buy intelligence from us on a meter'

We have governmental surveillance and we have surveillance capitalism. Surveillance capitalism works so well that governments are now very interested in the data they collect, which is alarming. Unfounded conspiracy theory: It's probably one of the reasons that governments don't seem interested in AI's regulation. If I had the proper equipment to run AI entirely local and efficiently so that the expenditure would justify it, I would.

I think they know it's a somewhat viable option and is part of the reason they're doing the hardware cartel/circlejerk thing.

Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I've seen in this thread:

[Thread #321 for this comm, first seen 30th May 2026, 09:50] [FAQ] [Full list] [Contact] [Source code]

P100s are dirt cheap on ebay fyi

Altman can try to hype up how everyones going to subscribe to them someday all the while their subscriber base is being eaten up by competitors.

https://www.wheresyoured.at/openai-projects-chatgpt-plus-subscriptions-to-drop-by-80-from-44-million-in-2025-to-9-million-in-2026-made-up-using-cheaper-subscriptions-somehow/

Local stuff. I still believe the small parameter, ~1B free local, ones will suffice for the vast majority of how people use LLMs and there's still going to be a few years of improvements there until investments dry up. Eventually I bet more and more phone companies will include one of these small ones out the box. Pretty much like a nice search engine that works offline like if you're out on a major hike. Cloud stuff, there'll be stuff like Proton's Lumo where they're taking free open weight stuff and piecing them together for users.

OpenAI's thing is they'll make up for falling subscribers with advertising. So pretty much we're advancing fast in the search engine race of the 90s/early aughts. We'll at least have Gemini. ChatGPT maybe ends up crashes in value someday and bought up by Microsoft or some other company. Deepseek, Qwen, Kimi. Claude like ChatGPT maybe survices or crashes and gets adsorbed by another company. Proton continue to exist as the company making AI products out of free stuff. Eventually the pace of improvements moves at a crawl and it's pointless to be paying for the best paywalled stuff. Just use the free stuff like how everyone mostly uses free search engines

Does anyone have a recommendation for a local model that can run well on a 5070 12GB? It pretty much would only get used for help with homelabbing and simple scripts.

My issue with the orphan-crushing machine isn't only that it's not in my children's bedroom

Sure but all these self hosted ais are still done by companies who used massive amounts of power and water to train it.

I started working toward self hosting LLM for my small company using ollama and opencode as agent But I realized a good model like GLM 5 requures 250GB of RAM and 24GB vram with a 4080?? I dont know, this is what the LLM told me itself.

I ended up using qwen-code2.7-7b-16k.

Currently the best thing I have is my laptop, 16GB ram, i7 9750H gtx1650

How do you guys selfhost? What models do you use that are actually good?

not gonna self host bullshit that wastes resources and makes me dumber.

You're still paying for electricity and a big part of the world is in a electricity crisis. "AI" has few real uses and LLMs are not one of them.

And are you sure you’r self-hosting or is it a plugin (that you’re self-hosting)? Also, I don’t invite SkyNet into my perimeter.

I'd like to draw a comparison: a cozy wood fire versus central heating. In the right time and place (eg camping in the woods), a wood fire is both very practical and very useful. Meanwhile, most homes built in the past 70+ years in the USA have central heating (or are somewhere that doesn't need heating at all) and the benefits are quite obvious: automatic temperature regulation, supplied by a utility, and low or no local emissions. And yet, there will still be rural homes that are heated exclusively by a wood stove, located in the middle of the living room, whose iron construction stores and radiates heat well after the fire has gone out.

Do I bemoan individual homes that use a wood fire? No, not really. The reality is that a grand, overwhelming majority of people don't have wood fires anymore. Even when air quality is poor, prohibiting wood fires in a few rural homes isn't exactly what would clear up the air.

Now, it would be a vastly different story if city-dwellers all had wood fires. When every home in a neighborhood is building and burning a wood fire, the results are disastrous: horrific PM2.5 in the air, soot coating everything, substantially reduced energy efficiency, and mass logging just to keep the wood supply. A mole-hill quickly becomes a mountain of problems when it's at scale.

So to that end, I would very much like to see commercial-scale AI reigned in, as the external costs have already gotten out of hand. What they have built is more correctly called a wildfire, not a wood fire. But where does that leave small-scale AI/LLM users? They can weigh the cost/benefits for themselves, provided that they don't harm other people or resources in the process.

But that brings us back to a cozy wood fire versus central heating: at small scale, a wood fire struggles to heat an entire modern American home (ie 2500 sq ft; or 232 sq m). Yet central heating does it with ease. Who then will be interested in this endeavor? Probably only those with a love for the camping aesthetic, and other enthusiasts.

At this point, it has become more clear what the utility of small LLM models is, and they do pale in comparison to larger LLM models. If small LLMs are what sensibly survives into the future, then that's essentially a cap on their capabilities, given a want to avoid burning the planet to run anything larger. The only way out would be for substantial developments in the energy efficiency of small LLM models, but that's not where the interest is.

No one is seeking to build a more efficient wood fire.

if you're selfhosting AI, make sure you at least firewall it off from the internet. many providers still send metrics back home that includes usage and content.