LLMs have been getting better at code generation over time. The problem is that AI boosters think that AI is and will get exponentially better over time, but in reality these increases are slowing down and increasing in cost, both in terms of training and inference (actually generating the output). Each advancement is smaller and costs much more than the previous one, creating a natural limit over time.

I’m not sure where the theoretical limitations are when it comes to code generation, but there’s also an economic limit. You can keep throwing resources at the problem, but the curve is exponential. For example, you can increase the size of the model or the context window by throwing more RAM at the problem, but there’s only so much you can cram into a system, assuming that we can even manufacture enough.

You're right, because that is a matter of technique and process, which our current global structure excels at accelerating.

It's just a matter of building a good enough list of things to check. The tech-debt of getting to that point shouldn't really be resting on the public though.