Qwen3.8-flash-next broke the vram requirement with its new architecture (which is a preview of their 4.0 architecture). You can run a 177b parameter model on 12gb VRAM, 64gb ram and 80gb SSD at 30tok/s and the speed scales almost linearly with increases to vram.
Incidentally, this was proven at almost the exact time all the calls for "regulating local models because they'll spread into the toasters" started.
You absolutely can have local models that run on consumer grade hardware, and the proprietary companies know it and are scared to death.