all 27 comments

sorted by: hot top controversial new old
[–] 66 points 7 months ago (8 children)

In a weird sort of way it does. Consider all of the following

  1. big companies are often incompetent and inefficient in a lot of ways
  2. The mozilla foundation has confirmed the security issues that Anthropic found were real
  3. Generally over the past few years, anthropic has some of the best, most reliable models
  4. Claude code has been kinda bad for a while
  5. Claude code has been mainly bot-written for a while as well. This can lead to functional, decent code that's still terrible in many ways as seen from the leak. Also it's entirely possible that bots are worse at detecting issues in bot written code. You could argue if they were good at it, they would be less likely to write those security issues in the first place?
  6. Anthropic could have very skilled ml engineers but mediocre software developers
  • source
  • hideshow 8 child comments
  • [–] 38 points 7 months ago (7 children)

    On the other hand: if their new tool is so great, why haven't they used it to fix Claude's security issues?

  • source
  • parent
  • hideshow 7 child comments
  • [–] 36 points 7 months ago (2 children)

    I've seen Claude prompts. They specifically asked it to create secure code.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 31 points 7 months ago

    Guy on TV this morning saying they've 'created a new species' and I'm like yeh, you've created a group of humans so dumb that no other human would be willing to have kids with them.

  • source
  • [–] 6 points 7 months ago

    Billions of dollars could have paid for a lot of security engineers. Wonder how many issues they could have found and fixed.

  • source
  • [–] 11 points 7 months ago

    Time for the Anthropic Apologists. I’ve noticed a lot of them recently.

  • source
  • [–] 4 points 7 months ago (1 child)

    Anthropic legitimately believes they've created a living being. Its fuckin weird.

  • source
  • hideshow 1 child comment
  • [–] 3 points 7 months ago

    I do enjoy telling people that OpenAI was literally a cult that believed they are/were building God. Musk, and Anthropic also believe that. Facebook and Google have mixed opinions on the subject. Microsoft is content just getting the bag. Oracle is also more down to earth only hoping to build a ubiquous survelice and secret policing system.

    Though a mixture of academic groups, a Chinese hedge fund, and a lot of volunteering motivated by Weird Sciencing themselves a girlfriend/daughter have also contributed a lot to the field.

  • source
  • parent
  • [–] 3 points 7 months ago (2 children)

    What is the 'instruction to mislead' referring to?

  • source
  • hideshow 2 child comments
  • [–] 13 points 7 months ago

    Likely the directive in the source code for users internal to Anthropic that the LLM should not make any reference to being an LLM or mention model names etc in commit messages or comments. So when they contribute code to external repos it's not immediately identifiable as LLM generated

  • source
  • parent
  • [–] 4 points 7 months ago (1 child)

    Scam Altman hallucinates more than his AI

  • source
  • hideshow 1 child comment
  • [–] 2 points 7 months ago

    How time and cost effective is this? Like I can dump 80k in tokens into finding an exploit or spend 80k to hire a guy who will probably take longer but not give me false positives.

  • source
  • [–] 2 points 7 months ago

    Well clearly people could use it to find security holes in their backend, since those def exist.

  • source
  • [–] 7 points 7 months ago (5 children)

    The company's shoddy opsec doesn't directly equate to the model's cabapilities. I am not one to believe anyone's hype, but I am not one to believe the AI anti-hype that goes on throughout Lemmy. A year ago, according to Lemmy, LLMs could never produce working code at scale. 6 months ago, according to Lemmy, LLMs could never produce working code that was secure enough to use in production. Now, Lemmy believes LLM can't be disruptive to cybersecurity as a whole.

    In 6 months I wonder what Lemmy will claim LLMs aren't capable of.

  • source
  • hideshow 5 child comments
  • [–] 3 points 7 months ago (1 child)

    I still have not seen evidence that LLMs can make it easier for engineers to produce working code at scale.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 7 months ago

    It's not evidence, but anecdotally i have two clients who have been working fully agentic for a good 6 months and they're smashing it. Even looking at it critically they haven't been able to find any obvious negative impact on code quality, product stability or performance or security.

    I think the secret sauce is that they weren't born with AI, they just integrated it into technical cultures that were already solid. In this context it does speed things up a bunch. Not a 100x multiplier or whatever dumb stuff they're pushing on Twitter, but you do get high velocity without burning out your team or sacrificing quality.

    And those 2 companies i just happen to know but they're nothing special. You take any boring software company from Paris or Berlin and i'm pretty sure you'll get the evidence you seek.

  • source
  • parent