the average person isn't going to use a locally hosted model, ever.
True.
I don’t think that’s true at all.
Platform vendors have already started shipping small LLMs for translation and for AI assistance. If you use Apple Intelligence you’re using local models to some extent for the smaller tasks. Consumer hardware is perfectly capable of running much larger models too if you give it enough RAM, and is only held back by RAM being expensive. My five year old laptop can run 30B-ish LLMs, and if RAM prices hadn’t exploded, then by now surely those 64 GB wouldn’t be an outlier and newer devices would be running more powerful models for “average people” too.
But prices for RAM will eventually come down again and hardware in general will keep becoming more capable. It’s weird to me to suggest anything else, considering that we went from 8 bit PCs with 64k of RAM to modern smartphones in my lifetime. Why would hardware stop improving exactly now?
The same applies to LLMs themselves. Qwen 3.8-27B beats expensive online models from two years ago in its stronger areas (e.g. coding). Why would that development suddenly stop now? There’s a hard limit here somewhere regarding the compression of knowledge (just based on information theory, not specific to AI) - which is why smaller models hallucinate more - but that doesn’t rule out e.g. improvements to how external knowledge can be integrated at runtime.
Accordingly, the AI models shipped with consumer devices will keep becoming more powerful and online models will be needed less and less for “average person” tasks.
One part of the AI bubble is the assumption that AI will remain compute intensive, which is what drives the data center buildout. But historically, new technologies have often started out as expensive and barely obtainable for average people and then became cheap and ubiquitous within fairly short time frames. It’s weird to me to suggest that LLMs will be an exception especially while also suggesting, at the same time, that the technology will accelerate research and development across the board.