There are thousands of resources for this information

https://youtu.be/ugKAkCVBcz0

https://www.fitmyllm.com/

https://meattacker.github.io/local-ai-model-recommender/best-local-ai-models-4gb-vram.html

There's scientific papers about that kind of degradation. Like this one from earlier this year.

In short, from what I read: No, I don't think it holds true. It'd be weird if it was some fixed number like 150k. If at all, I'd expect it to be a percentage of the maximum context length of a specific model. And it probably also depends on the exact model and how the attention and the parameters are set up.

It depends on the model. For newer stuff, and Qwen3.8 27b specifically, its Qwen Sparse Attention/Gated DeltaNet architecture does much better than a lot of previous models.

The old rule of thumb was that the middle of the context window can get lost in old models, but Qwen3.8 27b and Qwen 3.8 Flash Next work fine, even when using most of the 262k native tokens of context… but this long context requires 8GiB of VRAM just for the context, and this is with FP8 KV quantization.

I still don’t trust much more than 128k though, and most harnesses automatically compact if a request would exceed a certain threshold (e.g., OpenCode is the higher of 15% of max or 16k tokens for its buffer).

For smaller models 16-32k is probably fine unless you’re using a model that is extremely old, like llama3 or phi 2.5. But you’re going to struggle to get anything other than a simple chatbot with very short conversation on a 4GB card. Maybe light translation or document work, but don’t expect anything useful in coding or agentic work.

I believe Pocock only recommends going up to 100k now, last I heard. But for a lot of simpler tasks 10-20k will be plenty if you don't have a super long prompt + documentation for it to parse first. Your inputs are really what decide how much context you need.

I’ve experimented with a lot of that range of models. The sweet spot is 35-50k of context. Anything more and the response quality degrades, or the model times out trying to answer, or just keeps churning till it becomes nonsense. I had a model respond with “SimiSimiSimi” for 3 pages of content because I overloaded the context to about 130k tokens. Was really disappointing to see.

I was asking Kimi-K3 that very question and one of the possible things was a local model for VTT. I think looking at specialised models like that might be the best use case.

Another one I was thinking about last night was local semantic search through embeddings. I'm not sure if that's actually feasible, but it would certainly be useful for when you have terabytes of PDF.