Hell, that's the whole thing with these LLM-based business/product structures, isn't it? The models are very good at creating something that looks right, leading to people being absolutely blindsided when they fail to actually do the thing that boosters and salesmen pretended they were doing.

Given that these are statistical models that function probabilistically, it seem like the obvious attitude to take would be to assume it's a question of when they fail and do something wrong, rather than if. But accounting for that inevitability undermines most if not all of the actual economic value of these things because it turns out it takes just about as much time, effort, and skill to monitor and check these things as it would to just do the damn work yourself. But as soon as you start giving these things permissions to operate independently you are setting up a time bomb and putting duct tape over the timer. You will get fucked eventually.

it turns out that people who can’t code make terrible judges of how good ai is at coding

And now that you trust me, bonus me!

It makes every bad programmer into a 10x bad programmer (equivalent to 10 bad programmers).

What you’re saying is there will be lucrative billable hours to fix it when this all shakes out? I guess short term pain…

the most robust study in AI coding! cos they built it from toothpicks instead of toilet paper