you are viewing a single comment's thread
view the rest of the comments
[–] 10 points 14 hours ago (5 children)

Can anybody please explains it to a cybersecurity layman such as myself why that is? Did AI just exaggerate how serious those exploits are? Did it hallucinate the exploits whatsoever? Were the exploits uninteresting in the real-world hacking, where the human is still the weakest link?

  • source
  • hideshow 5 child comments
  • [–] 17 points 12 hours ago* (last edited 6 hours ago) (3 children)

    If I ask an LLM to review my code it will find a long list of issues. Some of them will be real, and some of them will look kind of like issues, but won't really be when you dig into them. Sometimes the issues will be kind of real, in the sense that there's a real best practice not being quite followed, but they don't actually rise to the level of a bug or vulnerability, and they're not worth fixing.

    If I fix that first list of issues, then ask an LLM to review my code again, it will, almost without fail, generate an equally long list of issues the second time. There might be some new real issues it missed in the first pass, but it will be much more slanted towards the "not quite a real issue" category, or duplicates of the "issues" I didn't fix.

    By the third round of review, almost everything it produces will be useless.

    I'm pretty sure this is what's going on with Project Glasswing.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 3 points 11 hours ago

    This is very true, and hilarious to me.

    The AI reviewer always says "approve with suggestions". Code that is logically flawed: approve with suggestions. Typo in a comment: approve with suggestions.

    But god forbid it deems my test coverage inadequate! "Don't approve" 🫠

  • source
  • parent