you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 10 hours ago* (last edited 10 hours ago)

The statistical distribution of errors in LLM produced code is Random Uniform, unlike with people-produced code were bugs tend to be more around certain areas because people have common sense so certain things are "obviously wrong" for them so they don't make mistakes there and end up doing more mistakes in complex logic and interaction between multiple parts.

This means that LLMs will makes really basic "nobody could possibly get this wrong" bugs right alongside incredibly complex ones.

So on top of the effects you describe from the way corporate environments work when it comes to IT security, you now have something that will randomly introduce some of the most basic security mistakes imaginable (for example pushing access credentials plain text into Github).

Then to add insult to injury, in a software development process where vibe-coding was introduced to reduce manpower costs (meaning that now there isn't enough manpower for humans to get to know and understand the codebase and do manual bug tracking and fixing) bugs are fixed by re-running the code generation, which doesn't actually reduces the number of bugs, just replaces old bugs with new bugs (as the probability of introducing bugs remains the same across code-generation runs).

Further, people actually produce fewer bugs as time goes on, both because of their increasing familiarity and understanding of the code base and because they themselves first learn to code better and eventually even start learning to design software in ways that reduce the probability of mistakes, whilst the AI doesn't learn and the best you can do to try and reduce bugs is to make sure there are certain instructions always in its context and hope for the best.

In summary, even on a "couldn't care less about IT Security" corporate environment LLMs increase the security flaws in the code base, never actually get better at doing less of it and, worse, are just as likely to produced stupidly basic and easy to exploit security flaws as they are complex and hard to exploit ones.

  • source
  • parent