One use case is auto mode for your local coding agent (automatic permission approval).
There is a lot of room for optimizing the inference for specific models. For example, when I switch llamacpp to Strata, I get to 45ts (granted on a slightly poorer quant).
You don't need to load the full model into VRAM. In my case it is offloaded into RAM. Some also get away with offloading to an NVME SSD.
Yup great combo. It baffles me why anyone one would favour to depend oneself on an online service instead
Actually they do. Through their findmy networks.
Yup worth a try. I got from 13ts to 21ts for Qwen 3.8 Flash Next on a RX 9070 XT 16GB. As always, the additional memory demand requires some compromise. For me reducing context from full 262k to 200k and reducing kv cache down to q8_0 was the best option. moe-cache-mib at 6544. Cache hit rate at ~70%.
Looks like the magnet is not buildin. It is attached to the case. Is that different on XTEink devices?
Sadly did not dig into the whole "the other side decides which device you are allowed to use" topic, a feature inherently build into passkeys.
It is almost as if they have no fucking clue who their customers are. Hilarious.
So who is going to jail now? I am sure a company is responsible for its actions, right?
Unless you're very unlucky, your worst case braking-related scenario is that you run the pads too low
Well guess what, when you're unlucky with those, better bring a spare wheel with you. 🫠
That is called opt-out though.
view more:
next ›