a coding LLM can code as well as a fresh CS grad.

For a couple of hundred lines of code, they might even be above average. When you split that into a couple of files or start branching out, they usually start to struggle.

after you give them a piece of advice once or twice, they stop making that same mistake.

That's a damn good observation. Learning only happens with re-training and that's wayyy cheaper when done in meat.