This is the big issue. LLMs are useful to me (to some degree) because I can tell when its answer is probably on the right track, and when it's bullshit. And still I've occasionally wasted time following it in the wrong direction. People with less experience or more trust in LLMs are much more likely to fall into that trap.

LLMs offer benefits and risks. You need to learn how to use it.

That's exactly the question, right? LLMs aren't a free skill up. They let you operate at your current level or maybe slightly above, but they let you iterate very quickly.

If you don't know how to write good code then how can you know if the AI nailed it, if you need to tweak the prompt and try over, or if you just need to fix a couple of things by hand?

(Below is just skippable anecdotes)

Couple of years ago, one of my junior devs submitted code to fix a security problem that frankly neither of us understood well. New team, new code base. The code was well structured and well written but there were some curious artifacts, like there was a specific value being hard-coded to a DTO and it didn't make sense to me that doing that was in any way security related.

So I quizzed him on it, and he quizzed the AI (we were remote so...) and insisted that this was correct. And when I asked for an explanation of why, it was just Gemini explaining that its hallucination was correct.

In the meanwhile, I looked into the issue, figured out that not only was the value incorrectly hardcoded into a model, but the fix didn't work either, and I figured out a proper fix.

This was, by the way, on a government contract which required a public trust clearance to access the code — which he'd pasted into an unauthorized LLM.

So I let him know the AI was wrong, gave some hints as to what a solution would be, and told him he'd broken the law and I wouldn't say anything but not to do that again. And so far as I could tell, he didn't, because after that he continued to submit nothing weirder than standard junior level code.

But he would've merged that. Frankly, the incuriousity about the code he'd been handed was concerning. You don't just accept code from a junior or LLM that you don't thoroughly understand. You have to reason about it and figure out what makes it a good solution.

Shit, a couple of years before that, before any LLMs I had a brilliant developer (smarter than me, at least) push a code change through while I was out on vacation. It was a three way dependency loop like A > B > C > A and it was challenging to reason about and frequently it was changing to even get running. Spring would sometimes fail to start because the requisite class couldn't be constructed.

He was the only one on the team who understood how the code worked, and he had to fix that shit every time tests broke or any time we had to interact with the delicate ballet of interdependencies. I would never have let that code go through, but once it was in and working it was difficult to roll back and break the thing that was working.

Two months later I replaced the code and refactored every damn dependency. It was probably a dozen classes not counting unit tests — but they were by far the worst because of how everything was structured and needed to be structured. He was miserable the entire time. Lesson learned.