New NVIDIA paper says AI models do not always need huge amounts of past conversation history (context) to work well. They show that regularly shortening the history (compaction) keeps accuracy almost the same on hard, long tasks.

you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 1 day ago

What have people found the best compaction scheme/harness to be for relatively short context? I've been trying to set up an agentic workflow and context management seems like the biggest source of jank and errors. Reassuring to see a paper like this though since the amount of context I can use and still have everything in vram is not that high.

  • source