The derived Elo is a great tool to isolate whether agent loops are actually "reasoning" or just brute-forcing the search space. It clearly proves that current agent scaling (via basic try-observe-reflect loops) quickly plateaus because it lacks the human capacity for abstract conceptual shifts and structural refactoring over long-horizon tasks. I believe the future of test-time compute in the agent era shouldn't just be about scaling trials or running more iterations; it should be about building architectures capable of hierarchical planning that can dynamically pivot their entire strategy when stuck.

AI is great at speed, not always direction.