256
“I’m calling it now, the adoption of AI agents into software development will be one of the most costly mistakes in the field’s history. Agents cannot program…” (geohot.github.io)
submitted 4 months ago by to c/webdev@programming.dev
83 comments fedilink hide all child comments
💯 3
 

I’m calling it now, the adoption of AI agents into software development will be one of the most costly mistakes in the field’s history. Agents cannot program, and it’s taking longer and longer to realize that they can’t. They are a highly sophisticated statistical model designed to mimic the distribution of programming. The output is broken, but in a way that’s getting harder and harder to detect. Which is exactly what you’d expect from an increasingly accurate statistical model.

you are viewing a single comment's thread
view the rest of the comments
[–] 11 points 4 months ago (11 children)

But you're forgetting the key difference that makes it so much worse - we can fix human mistakes especially if we can talk to the human to figure out how. With an llm we have no external reference, only poorly designed code where the comments are there to guide the writing, not describe what was written. So it's much harder to debug an output, and the llm cannot be trusted to clean it up either.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 11 child comments
    •
  • [–] 9 points 4 months ago (1 child)

    A human can be held responsible. A machine cannot. If the machine writes bad code, and someone gets injured or killed because of it, who takes responsibility?

    I state again: a machine cannot be held responsible.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [+] -6 points 4 months ago (8 children)

    You can totally fix AI-written code with AI. You tell it something is wrong, it tries to fix it.

    I did a recent experiment with AI writing a document format converter and that's exactly what I did. It wrote some code, I checked the output, found a formatting issue or similar, asked it to fix it, repeat. It works unreasonably well and with Fable the final code isn't even bad.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 8 child comments
    •
  • [–] 6 points 4 months ago (5 children)

    You can fix problems, if you know they are there and there is a model of that problem being fixed.

    You can't fix problems you don't know are there, or do not have modeling.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 5 child comments
    •
  • [+] -6 points 4 months ago (4 children)

    That's true for humans too surely? How would anyone fix problems they don't know exist?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 8 points 4 months ago* (last edited 4 months ago) (2 children)

    Humans generally don't hallucinate libraries or documentation. If there is a bug or error on a human maintaine repo the human in charge will generally know what went wrong and how to fix it, the AI will just gaslight your ass because the AI has no idea.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 3 points 4 months ago

    To add to the other response - it is much more difficult to work with Ai to debug inconsistent issues or similar unless you can understand the code and step through with a debugger to check for race conditions or similar.

    Recently I was working with an Ai tool for some c code that depending on machine ran wildly differently. The Ai was unable to identify any issues, and kept recommending fixes for hardcoding values or similar that I had to revert. The fix ended up needing to use valgrind to create a different enough environment to see how a race condition was made to properly have one async call delay for the other.

    AI can be powerful, and humans can be dumb. But if the code was human made, I would not have needed 3 hours to find a problem, and I wouldn't have tried to turn to AI for a simple fix because I'd know what I was looking for to start with.

  • permalink
  • fedilink
  • source
  • parent