Key Takeaways:

Seven months into Project Glasswing, VulnCheck has tracked 442 CVEs attributable to Anthropic, roughly 0.5% of all CVEs published this year, with only two identified as exploited in the wild. Of Anthropic’s 37,966 reported findings, only 7,349 (19.4%) have been made public and just 217 have been marked fixed. Since September, Anthropic has added nearly 12,000 findings while recording just 15 additional fixes, widening the gap between discovery and remediation. Roughly 1,654 findings (4.4%) have been marked as duplicates, nearly eight times the number fixed, and another 419 have been withdrawn. Anthropic’s claimed true positive rate has risen from 90.8% to 94.2%, raising questions about how it accounts for duplicates and withdrawn findings. Anthropic’s OSS Scanner shifts validation work to maintainers, with curl rejecting 19 of 34 published reports as vulnerabilities, confirming six as low severity and leaving nine awaiting assessment. Also putting Anthropic's true positive rates into question.

1 comment

sorted by: hot top controversial new old
[–] 1 point 4 days ago*

What a stupid article, of course all these insane vulnerabilities aren't going to be made public until at minimum some substantial time has passed after they've been fixed. Yes it's overwhelming but that's the nature of needing to confront all your security tech debt at once

  • source
  • 12 comments in c/technology@lemmy.world view that post
    [–] 33 points 6 days ago (2 children)

    A place I worked at fell for this AI marketing doomerism around vulnerabilitues hook line and sinker. The tool lit up the organisation's infrastructure like a Christmas tree labelling thousands of configuration issues and potential exploits, yet only a tiny fraction of was ever truly exploitable from the open internet. Millions blown on this junk and hundreds of hours wasted triaging meaningless reports all to appease an LLM that doesn't know jack shit about threat modelling. But hey executives said scary number went down and that's good right?

  • source
  • hideshow 2 child comments
  • [–] 30 points 6 days ago* (last edited 6 days ago)

    Anthropic

    an overhyped marketing campaign.

    Of course it was.

  • source
  • [–] 34 points 6 days ago

    They use this crap just for marketing. Now, the marketing turned bad.

  • source
  • [–] 11 points 6 days ago (6 children)

    Can anybody please explains it to a cybersecurity layman such as myself why that is? Did AI just exaggerate how serious those exploits are? Did it hallucinate the exploits whatsoever? Were the exploits uninteresting in the real-world hacking, where the human is still the weakest link?

  • source
  • hideshow 6 child comments
  • [–] 22 points 6 days ago* (last edited 6 days ago) (4 children)

    If I ask an LLM to review my code it will find a long list of issues. Some of them will be real, and some of them will look kind of like issues, but won't really be when you dig into them. Sometimes the issues will be kind of real, in the sense that there's a real best practice not being quite followed, but they don't actually rise to the level of a bug or vulnerability, and they're not worth fixing.

    If I fix that first list of issues, then ask an LLM to review my code again, it will, almost without fail, generate an equally long list of issues the second time. There might be some new real issues it missed in the first pass, but it will be much more slanted towards the "not quite a real issue" category, or duplicates of the "issues" I didn't fix.

    By the third round of review, almost everything it produces will be useless.

    I'm pretty sure this is what's going on with Project Glasswing.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 3 points 6 days ago

    This is very true, and hilarious to me.

    The AI reviewer always says "approve with suggestions". Code that is logically flawed: approve with suggestions. Typo in a comment: approve with suggestions.

    But god forbid it deems my test coverage inadequate! "Don't approve" 🫠

  • source
  • parent