Hacker reverse engineers his Samsung TV and finds on-device AI listening for your dog and baby, watching what you watch, and borrowing your soundbar's mic if the TV doesn't have one. Then OpenAI banned him for reverse-engineering it.

Source:

you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 1 week ago (2 children)

I guess the question is, what gets priority, RAM or GPU. I can get a dual 192GB 6000 nvidia with 256GB of RAM for $50k, but that still doesn't get me the capability of top models. Or I could get a 1TB of RAM tower server with maybe one 96GB nvidia 6000 for about the same. Does the RAM capacity mean I can fit the larger model but it's just going to be slow?

  • source
  • parent
  • hideshow 2 child comments
  • [–] 3 points 1 week ago

    With Strata, things have gotten more flexible. Similar optimizations are coming to llama.cpp eventually.

    https://github.com/Niko1221/Strata/blob/main/docs/MODELS.md#pick-by-memory

    It says Qwen 3.8 Flash Next can run with just 8GB VRAM and 96GB RAM for a good quant of it. This is a really good model, and the new version 4 should be coming in the next month or so, probably the same size but smarter.

    DeepSeek v4.1 Flash is probably even smarter, says it needs 12+GB VRAM and 64GB of RAM. For decent speed I think you'll want at least 32GB VRAM and 256GB RAM for this one. 96GB VRAM should be very good.

    You're gonna have big diminishing returns chasing for even bigger models, partly because Strata doesn't support anything bigger and llama.cpp is currently not nearly as optimized. There might be an inference engine I don't know about for the biggest models.

  • source
  • parent
  • That's a good question and it's hard to say for sure. There've been some advancements this year in offloading, using both VRAM and system RAM together. With MoE models and predictive loading you can get pretty good results that way...sometimes. I've never tried it myself, though I really should because my 24GB GPU is a serious bottleneck.

    You could also think about getting a 512GB Mac Studio (coming soon). Not as fast as a GPU, but maybe faster than a GPU/DDR5 mix? And definitely cheaper per GB.

    I realize now that the truly biggest models need way more than $40k of hardware. The biggest Qwen3.8 is 2.4 TRILLION parameters. Yikes.

  • source
  • parent