7
It Was the Harness, Not the Model — 90% of It (blog.herlein.com)
submitted 1 month ago by to c/aicoding@programming.dev
comment fedilink hide all child comments
 

I ran five different coding agents at the same local model, on the same task, with the same frozen test suite — and then I counted why they failed. The answer wasn’t subtle. About 90% of the failures were harness problems. Only about 10% were the model.

no comments (yet)

sorted by: hot top controversial new old
there doesn't seem to be anything here