This is a problem for a ton of developers today. In my last round of interviews I heard a lot of consternation from people who were perfectly capable during the code review about their newly discovered inability to write code without AI.

The code review has always been a bigger hurdle than the writing in the past.

Entirely possible I’ll eat crow and look back on this as a major mistake.

I think the issue would be with complacency. After reviewing so many changes it could be easy to say “eh - it’s probably good” and merge it. I don’t have confidence in it’s output the first, second, or third time.

I think another issue is if I was using it in a domain or something I wasn’t familiar with like hardware programming. That’d be a bit like the blind leading the blind.

I'm currently working on fixing stuff like this as an external consultant. Like OP, people confidently assumed the LLMs could handle writing the code.

Oh boy, were they wrong. As soon as the code was pushed to production, the entire stack collapsed.

I'm curious about your setup. We're a small company trying to up productivity until we can hire more people and so far we've been trying some agents that are very good for some tasks but we're still writing a lot of code.

Dang, lucky. Besides the most boilerplate of stuff I try do I get this set up to work. It gives a 60% but also 10% wrong start most of the times on a new project or feature. Helpful for debugging like 70% of the time. Otherwise I'm stilling monkeying around anytime I need to putter anything

I regularly write code.

My customer gave the go-ahead to use LLM in our project very recently. We'll be trying it out. I'm interested to scope out its use and limitations especially. I'm skeptical it will increase efficiency for me overall. The project is too complex, my/our requirement on quality too high, and I'm thorough to the last var name and code formatting for readability and obviousness. I'm not sure whether I could find it acceptable to compromise on those.

Between customer communication, planning, review-prep, guiding and helping my team members, and doing reviews, and other tasks within the company, time for my own work can be reduced by a lot. Still, I have tasks I work on, and that includes coding.

I work in an industry where LLMs aren't helpful. And I've already automated away most of what they might have helped with.

Personally I don't like using LLMs for anything. Hell, I barely tolerate tools like lint. Gimme Notepad++ for my actual development work and I'm happy.

Gotcha – I have no doubts that LLMs can steer things to shit at breakneck speeds.

I certainly wouldn’t mind some more (competent) employees.

I don't think that'd be much good when the way the specs are written are not based purely on technical merits. We also aren't paid to test hardware - we have hardware because we develop software to test hardware on. Knowing how to navigate those waters is the key skill. Once I've got a solution that'll work getting it implemented is relatively trivial as our approach is extremely atomic.

In general what you describe sounds like a tremendous amount more overhead than we currently have for little to no gain. What I could do with is a few more engineers that I could train up to have sufficient contextual knowledge, not another junior to babysit. I trained one up and he's tremendously useful - apart from when he leans too heavily on LLMs. That cost a side project two months of unnecessary faff and sapped team morale massively in the process. I ended up dragging the damn thing over the finish line after he refactored it into something that was exhausting to work with.

This is about the same behavior I’m accustomed to. I will say that my current work is more greenfield at the moment.

First plan is about 60% there. And then we have a few iterations to get that in good shape.

Once the plan is together I send that to a virtual machine to implement the code with low/no supervision.

Then it goes into a draft PR that I will review and provide further guidance on changes or updates. Those iterations will happen on either my virtual or local machine, depending on how much the work is.

I could imagine It would be pretty difficult with hardware, where you have to compile, and potentially transfer to a chip to run or test further. But I will say, if you can give an LLM access to the whole loop of write, compile, test and reading error logs the results can sometimes be impressive.

Chatgpt8? Why do we need 8 when 5 was supposed to be utterly revolutionary?

By ChatGPT8 when you don’t need to review it anymore for the level he’s at.

Unless it's the most rudimentary logic I tend to have to hold the LLM's hand through the entire design. Sometimes that involves breaking the damn thing's fingers.

I mostly use them to write stuff that I can do but hate because the syntax is a faff (argparsers, docstrings, shite like that). There's just too much contextual knowledge needed for them to be much use with the codebase I work with, much of which is not in the codebase so much as in the metatextual knowledge needed. I write a lot of software for testing hardware and there's a needle that has to be threaded - the specifications are not bulletproof, the hardware being tested failing may be the desired outcome (so code that passes on it is inherently wrong), and there are various limitations and priorities that inform test approach.

The actual code I write is fairly minimal but the knowledge needed to write it is extensive.

Occasionally I'll take on side projects at work to get my hands dirty on something more involved but simpler.

Yup. I guess it’s also worth noting my past couple jobs have been from my network - so the interviews have been more of a informal behavioral/culture fit. But I won’t pretend my network will always give me that flexibility.

I would say I’m the main motivator. But it is motivated to fulfill deadlines. It’s not something that I hide from my boss or coworkers.

You won't be the only one, surely interviews are going to have to change once no one can do white boarding anymore!

A recent JetBrains survey I saw found that 85% of devs are using AI in some capacity.

API/Data Engineering for SAAS products. No one will die if I do something dumb - it’ll just cost money and/or reputation if things go very bad.

I’m on the fence here.

I’m still steering and guiding the design - with knowledge gained over many products, features and incidents - and am reviewing it.

To scale higher such that I could be replaced then I think the change sets would have to be smaller and/or we have perfected bug/incident detection and remediation such that we can bypass human review of the code.

I certainly foresee this happening and/or my ability to perform any future white boarding interviews adequately.

if you ever vibecoded, you'd know, that using an agent doesn't mean you accept every suggestion. it does not also mean you edit the suggestions. it means you continue guiding the agent until it comes up with a "perfect" solution that may even be accepted without any edits. so if you are a good "prompter", given a good model you can be very very efficient without writing even a line of code. does this mean you can be replaced by the said model? absolutely not.

I guess they get the corrections and adjustments done by telling the LLM what corrections and adjustments to make?

What kind of work do you do? Every time I read stuff like this, I find it hard to believe, but maybe the code/languages/frameworks I'm using just aren't as easy for LLMs as what other people are using. The results I've had trying to get it to write C++ have been atrocious.

I'm not against AI code assistance, and I like keeping an open mind. For the moment though, the only success I've had is with using it to explain some feature or API faster than manually looking up the documentation. But actually reading the docs, I've found, helps me remember things better.

Sounds like you're a technical manager now -- or, at least, most of the way there -- just with LLMs as your reports instead of junior devs...

I still write my code by hand in xed. I'm not exactly anti-AI (my feelings are mixed); I'm just the kind of programmer who wasn't using IDEs even before the LLM craze started...

If they actually havent written any code then that means they didnt have to correct anything which means the LLM doesnt actually need them much if at all. Im assuming the post title is simply not true and they did indeed make some corrections and adjustments.

I see this all the time with CAPL, copilot gets close but messes so much up its not worth it. Not enough on stack overflow to train it, and until recently the docs weren't even available online to crawl.