158
Linux kernel czar says AI bug reports aren't slop anymore (www.theregister.com)
submitted 7 months ago by to c/technology@lemmy.world
99 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 34 points 7 months ago (6 children)

As someone who has to sift through other people's LLM code every day at my job I can confirm it has definitely not gotten better in the past three months

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 6 points 7 months ago* (last edited 7 months ago) (3 children)

    We require you to submit markdown plan before working on a feature, which must have full context, scope, implementation details. Also verification tests mardown file of happy path and critical failure modes that would affect customer, and how tests were performed. Must be checked in with the commit. More complex, large features require UML diagrams of architecture, sequences, etc. to be checked in too.

    If your plan or verification docs have wrong context, missing obvious implementation flaws, bad coupling, architecture, interfaces, boundary conditions, missing test cases, etc then PR rejected.

    Every developer's performance is judged as a systems engineer. Thoughtless features without systems docs and continued lack of improvement in your systems thinking gets you PIPed.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 10 points 7 months ago (2 children)

    That's the thing though. Even if the code is good, the plans are good, the outputs are good, etc, it still devolves into chaos after some time.

    If you use AI to generate a bunch of code you then don't internalize it as if you wrote it. You miss out on reuse patterns and implementation details which are harder to catch in review than they are in implementation. Additionally, you don't have anyone who knows the code like the back of their hand because (even if supervised) a person didn't write the code, they just looked over it for correctness, and maybe modified it a little bit.

    It's the same reason why sometimes handwritten notes can be better for learning than typed notes. Yeah one is faster, but the intentionality of slowing down and paying attention to little details goes a long way making code last longer.

    There's maybe something to be said about using LLMs as a sort of sanity check code reviewer to catch minor mistakes before passing it on to a real human for actual review, but I definitely see it as harmful for anything actually "generative"

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 7 months ago* (last edited 7 months ago)

    That's a good point. We've been using the UML diagrams as a tool to catch behavioral red flags, but the reuse and implementation details of that are left undefined.

    Maybe the answer lies in also explicitly spending a few passes focusing on code health, explainability, maintainability (while keeping brain on). This is something I go through at end and then retry verification tests, but not something we explicitly require in our process at the moment.

    But in the end you have to know what good looks like and be able to call bullshit. Hmm I guess strong first principles are still the foundation of being good at something, no matter how the tools change. And practice, feedback, and constraint exposure are what turn that into actual effectiveness.

  • permalink
  • fedilink
  • source
  • parent