78
AI Software Development Is Near-Impossible (www.youtube.com)
submitted 4 months ago by to c/programming@programming.dev
81 comments fedilink hide all child comments
 

Support this channel on Patreon ► https://www.patreon.com/zoranhorvatGenerative AI can write code, but it cannot develop software on its own. Here is why the...

you are viewing a single comment's thread
view the rest of the comments
[–] 9 points 4 months ago (15 children)

Tell him this: If you’re writing that much boilerplate (grunt work), you’re probably doing something wrong. If AI can write your normal work, you really should be learning, so not using AI at all.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 15 child comments
    •
  • [–] 13 points 4 months ago (14 children)

    Actually I'm not going to tell a dev with decades of experience that an internet rando thinks he's doing his job wrong lol.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 14 child comments
    •
  • [–] 8 points 4 months ago* (11 children)

    K. It’s astounding to me that after decades in the business, he’s still writing code that can be written by basically an intern.

    When I try to get AI to write my code, it stumbles all over itself just failing to understand the simplest of my libraries. (Literally it made multiple mistakes using the fucking tokenizer library I wrote that is so simple, an intern would have no problem with it.)

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 11 child comments
    •
  • [–] -1 points 4 months ago (2 children)

    Yeah, so if you are a senior dev, the easy work just disappears? Like if you are an experienced carpenter, you will never have to drive nails again, because that's too easy!

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 7 points 4 months ago

    If the majority of your time is spent on the easy work, you’re probably not a senior dev. If you are, maybe you shouldn’t be.

    Yes, we still have to do the easy work, but it doesn’t take much time. I’d rather do it myself and know that it’s done right, than let an AI do it and have to fix all of its mistakes. In my experience, it takes the same amount of time.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 3 points 4 months ago

    any senior carpenter that is doing the easy work has a jig they custom built to do it in a tenth the time. same for a dev. templates in your ide, the use of specific libraries that make developing tests 10x faster than a normal junit test. Whatever it is, if you're a dev with decades of experience and you haven't gotten to the point that the 'grunt work' isn't a few seconds of time then you're not a good dev, you're a shit dev who sat on their ass for decades doing nothing.

    And now AI has given you the chance to be even better at that, sitting on your ass doing nothing.

  • permalink
  • fedilink
  • source
  • parent
  • [–] -5 points 4 months ago (7 children)

    he’s still writing code that can be written by basically an intern He usually does full-stack projects by himself, so he has to do everything. And he's using AI to do what interns could do. I've dabbled a little using VSCode AI myself to refactor and upgrade a couple hobby projects, and it didn't "stumble all over itself" at all. In fact it conversed with me like an intern or colleague would, and made many proposals I agreed with. There are ways to craft your prompts that make AI work better. Maybe that's your problem I dunno.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 7 child comments
    •
  • [–] 4 points 4 months ago* (6 children)

    Even if it does write code that works, it usually (about 50% of the time in my experience) has bugs, and sometimes those bugs can be really difficult to spot. For me, it has never saved me any time. I’m either fixing something it doesn’t know how to do correctly, or going over its code with a fine tooth comb because when it says, “this is production ready code, with no bugs,” it’s usually wrong. That takes a lot of time. It’s easier for me to just write the code correctly myself.

    Admittedly, I haven’t used that new model that Anthropic revoked access to the public to recently. Maybe that one is good enough for government work.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 1 point 4 months ago (1 child)

    Fable was "just ask and get it done" quality level, but really I don't get THAT much bugs - about the same amount that I see irl developers do: get a new feature, find 5 problems, get them fixed, find one more, done. As a recommendation - try to alleviate the biggest problems that ai models have:

    • overconfidence - skipping wrong stuff "because they have a note saved that it works" or losing the point where they stopped after a session broke, then making things up. Test Driven Development solves the majority of problems like that - when Claude writes tests first, then it's not able to bullshit me that the job is done when "everything is red"
    • even with large contexts, they run out and stuff gets lost. So if you're not doing something really compact, make your ai document everything, document the feature they are working on now, make it then offload it to permanent doc when that piece is finished. When the ai will fuck up next time, you can tell it to "go read some docs" and most of the times it will work
  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 3 points 4 months ago*

    I’m glad it works for you, but it simply does not work for me. Maybe you could try yourself on some of my libraries, because I have never gotten it to save me any time. It’s just spending money and making the work less fun for no reason. Oh, also not having the copyrights to the things that go in my code base, don’t forget that.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 1 point 4 months ago (3 children)

    When I first tried it I felt lost, but after watching a couple videos about writing good prompts I had no trouble getting it to produce perfectly good code that did what I wanted. Your mileage may vary.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 3 points 4 months ago (2 children)

    Honestly, I think what you consider „good code“ is just shifted from what the previous commenter considers „good code“. Prompting is about giving enough information so the AI can solve the task without needing to reconstruct a lot of context. Most people using AI somewhat regularly will have figured out to write good enough prompts. I‘ve never seen AI generate perfectly good code beyond hello world and the fibonacci sequence. And by perfectly good I mean I wouldn’t change beyond 30% of what it produces, which is not a high bar.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 4 months ago (1 child)

    The last thing I had it do was create a filesystem using Discord as the storage medium. Don't ask me to justify the approach or tell me it breaks Discord's rules - that's irrelevant to my point. It was a fun idea to explore a way to host a bot for public use without also hosting other people's data. The data is pretty minimal but does involve several small tables with relational links. I thought it would be interesting to store it as a few Discord messages in a private channel.

    As sample data I gave the AI the JSON file I had been using locally, and discussed various aspects of the design with it. It came up with the scheme of splitting up the data using a channel for each table and a message for each row. I don't remember the interaction in detail but it asked me a bunch of questions - for example, it commented that Discord's limit of 2000 characters per message didn't seem to be a problem given the sample data, but wanting me to confirm. I had it generate the code as a node module with complete CRUD functionality, and refactor my existing code to use the module instead of the JSON file. What it gave me worked perfectly right away.

    Now of course this isn't "Write me an accounting system," but it's far more complex than Hello World or Fibonnacci. Whether or not you code with AI is your choice, but deciding my notion of quality isn't up to someone else's because my results aren't "AI sucks" is just pure denialism.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 4 months ago

    Honestly, I would argue about the complexity of it. The discord API is straightforward, and what you're describing doesn't sound that complicated.

    And we have seen fairly complex stuff that AI did, but if it does pull it off somehow, the next question is quality.

    I just read a bit of AI code a few minutes ago, and it didn't take long to step over a chain of if / else if statements where only the first if statement if reachable, the rest is not. The code should've been one if statement followed by a match statement. Not sure why the LLM thought "combine" two logically completely separate statements into one. And that was in 1.5kloc it generated, so not much. There's probably even more, I just read a fraction of it.

  • permalink
  • fedilink
  • source
  • parent