you are viewing a single comment's thread
view the rest of the comments
[–] 8 points 1 day ago (6 children)

"Cleaning up" seems a little generous. It's finding them, but someone still has to patch them.

  • source
  • parent
  • hideshow 6 child comments
  • [–] 3 points 13 hours ago (1 child)

    "Finding" is also rather generous, going by numbers from Anthropic and from the Linux kernel, things like Mythos are wrong 95% of the time.

    They're much easier to run though, compared to more reliable tools. So even being wrong 95% of the time wins out if you've got 100x more people running them.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 1 hour ago

    A 95% false positive rate isn't actually that big a deal if you have a good way to test the claims. I wonder if you could set it up CtF-style, where you don't get notified until the LLM can write to a specific file on the target box.

  • source
  • parent
  • [–] 1 point 1 day ago (3 children)

    Most vulnerability patches are a few lines of code. Finding it is the hard part.

    Even then, the AI session that found it probably already has a solution to fix it.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 4 points 19 hours ago (2 children)

    In my experience, the LLM usually proposes a solution, but it's a crapshoot whether it works well. And plenty of bugfixes go well beyond a few lines, especially in older code.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 2 points 9 hours ago (1 child)

    I use this shit on a daily basis at work. They work well, as long as you're choosing the right models, making the right engineering decisions, and have a good set of baseline rules.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 1 hour ago

    Do you have a guide on which baseline rules are good? I use Claude Opus 5.5 at work and while it's definitely more helpful than not having it, it also does some incredibly baffling things that a human would realize immediately are unproductive.

  • source
  • parent