Hacker reverse engineers his Samsung TV and finds on-device AI listening for your dog and baby, watching what you watch, and borrowing your soundbar's mic if the TV doesn't have one. Then OpenAI banned him for reverse-engineering it.
Source:
I can run Deepseek V4 (or V4.1 with some hacking/dev branches) on a single 3090 at 20 tokens/sec text gen. That’s straight-up better than OpenAI free tier, or the cheap Claude API, not to speak of bonuses from local running (like constrained sampling/grammar, custom sampling, context swapping, raw completion syntax, SGLang-style structured inference, task vectors, no censoring…)
I know 1x 3090 + RAM is kinda cost prohibitive now. But before the RAMpocalypse, that was not an unreasonable home server.
No, it’s not SOTA, but it’s good enough.
Now, the truly mad homelabbers?
They can run 1T models in a similar hybrid inference configuration, with an AMD EPYC server and some janky GPUs. MiMo Pro, GLM, those kind of models get you very close to Claude in most workloads. The gap is maybe months, not years.
And while Claude is good, honestly… OpenAI models are kinda shit. They are deep fried. I’ve never subscribed to them, and I never liked any of the bits I sampled since GPT 4, as they tend to be so sycophantic and slopped.
Are big servers cost effective? Hell no. It makes zero financial sense. There’s no reason to try and host that class of model when they are so dirt cheap over API.
Do I endorse mad homelabs anyway? Absolutely.