Comeback? Tapes never went away. Most long term cold storage offerings are backed by tape
A calculator hasn't changed much since 1990s. The world has moved on, but it still does the one thing cheaply and reliably. We had hiccups in beginning (expense, limited precision, etc.) and now we have phones and computers to eclipse it, but we still have calculators being produced, sold and used. Once you found a model to fit a use case, it can freeze till you can justify the expense of upgrading the processes around it.
Wrt your questions: I love that you are asking the right ones. You can delve into deep infer for more details on diff tradeoffs between batching and latency for the specific DeepSeek case. In general, more users allow larger batches and higher efficiency as well as better amortization.
S curves are genuine problems, though engineers are pretty good at finding something else to offset the slowdown. Computer chips, batteries, solar panels, fiber optics have been pretty good examples of scaling in hardware space. Signalling (Mobile LTE, WiFi, optical trancievers), games, etc. have shown scaling in mixed hw and SW space.
Scaling might stop for one thing, but then people focus on a diff item to still continue the forward progress. What would that look for models? I don't have a strong idea, but we have already seen MoE, KV Cache quantization, RL Post Training, etc. as major knobs beyond just scaling model size, with each new knob showcasing the same logistics curve behavior. From HW front, there has been incessant march since 70s, where we have seen multiple plateaus and the engineers still moving the frontier ahead. Freq Wall was hit at 2GHz and we have 6 GHz chips now, at a slower pace but still. Memory Wall was hit and yet HAMR and multi level cells have given us densities like never before.
More than mass surveillance, they are pedalling for decision making via AI. Running detection via vision models 24x7 is super cheap, most of them can run on the camera chip itself.
It's the larger models that are supposed to turn govt into the biggest sucker of SaaS (surveillance as a subscription :( ). They lower the morality bar (by hiding details. Just ask a model and it'll find data so you dont feel guilty of trawling through people lives) as well as intelligence bar (fewer vocal employees who could raise a voice)
A lot of local models are already at a place where we don't need them to be better, they are doing 90% of what's needed to significantly change the landscape. Continual growth is not required. This is delusional to think otherwise.
Local AI HW seems to be roughly 2x perf in 3 years. In 10 years, we should be running at roughly a magnitude more, either same model faster or larger models at same rate.
DeepSeek has shown that with the right inference machinery, they make more than 35$/hour/gpu, so even non local models can be profitable since renting a GPU is cheaper than this. The main moat is the inference kernels: more efficient= more profit.
Inference HW seems to be improving roughly 2x in 2 years or faster. In 10 years, that's be 32x faster, roughly the gap between a rack vs a single GPU.
Training isn't that expensive for smaller models. For frontier model, Xiaomi live trained one in a couple million dollars. That's expensive but not impossible for a company to do one frontier model a year, or a few smaller ones per year. Beyond all this LLM hype, the older CNN are still living a good life as experts in their small domain.
What's not profitable is doing frontier models every month (like a dick measuring contest), or giving inference for free (via free plan or subsidized subscriptions) or spending on marketing and Superbowl ads or booking gpus to scale by 10x each year or quarter (spot prices are roughly 2x more than pre booked at this scale).
This is something people aren't seeing. The Internet bubble burst as well, and we alwere using more internet by Y2k than ever.
Even if AI bubble bursts, say in 2027, I'd wager that we'd be using more LLM on 2037 than ever
AAA might be more portable though Texas might loose them in a strong gust of wind
Nuclear has lots of regulations so there had been little to no research in scaling it up compared to the vast sums of private RnD that panels and batteries have seen.
Only dark clouds reduce solar panel outputs. Panels don't care if the light is hitting directly from the sun or diffused by cloud, as long as sum of incoming radiation over surface is high, you get your energy
EULA has T&C that the first users agree to when they fast forward with the accept accept accept accept screens
Never seen anyone try to drive established successful businesses into ground this hard. (Trump can bankrupt anything but that's Trump being Trump)
Tesla cars were coveted and now any new Tesla carries a fascist undercurrent.
Twitter was an ok ok platform and he spurred the entire fediverse (and bluesky) into overdrive.
Starlink was a monopoly and cruising to victory and now multiple countries have decided to compete and intentionally not sell it spectrum.