It depends on the model. For newer stuff, and Qwen3.8 27b specifically, its Qwen Sparse Attention/Gated DeltaNet architecture does much better than a lot of previous models.

The old rule of thumb was that the middle of the context window can get lost in old models, but Qwen3.8 27b and Qwen 3.8 Flash Next work fine, even when using most of the 262k native tokens of context… but this long context requires 8GiB of VRAM just for the context, and this is with FP8 KV quantization.

I still don’t trust much more than 128k though, and most harnesses automatically compact if a request would exceed a certain threshold (e.g., OpenCode is the higher of 15% of max or 16k tokens for its buffer).

For smaller models 16-32k is probably fine unless you’re using a model that is extremely old, like llama3 or phi 2.5. But you’re going to struggle to get anything other than a simple chatbot with very short conversation on a 4GB card. Maybe light translation or document work, but don’t expect anything useful in coding or agentic work.