top 65 comments

sorted by: hot top controversial new old
[–] 296 points 7 months ago (17 children)

Perhaps the most discussed technical detail is the "Undercover Mode." This feature reveals that Anthropic uses Claude Code for "stealth" contributions to public open-source repositories.

The system prompt discovered in the leak explicitly warns the model: "You are operating UNDERCOVER... Your commit messages... MUST NOT contain ANY Anthropic-internal information. Do not blow your cover."

Laws should have been put in place years ago to make it so that AI usage needs to be explicitly declared.

  • source
  • hideshow 17 child comments
  • [–] 125 points 7 months ago (1 child)

    The system prompt discovered in the leak explicitly warns the model: "You are operating UNDERCOVER... Your commit messages... MUST NOT contain ANY Anthropic-internal information. Do not blow your cover."

    This is so incredibly stupid.

    You've tried security.

    You've tried security through obscurity.

    Now try security through giving instructions to an LLM via a system prompt to not blow its cover.

  • source
  • parent
  • hideshow 1 child comment
  • load more comments (1 reply)
  • [–] 14 points 7 months ago

    In Europe we have the AI act which, as of August, will introduce some form of transparency obligations. Not perfect obviously but a start. Probably will not be followed by the rest of the world though so like GDPR it will be forcibly eroded by other’s interests through lobbying but at least we try.

  • source
  • parent
  • [–] 9 points 7 months ago

    Haven't read the article and have a limited knowledge of ai, but I wonder if they do this for reinforcement learning: So OSS PR responses can be used to label different weights and models. Using even more free labor to train their models.

  • source
  • parent
  • [–] 1 point 7 months ago

    What internal info are they worried about leaking in a commit message? If you don’t want it to add the standard Claude attribution, you can completely disable it in the settings, or just write your own commit messages.

  • source
  • parent
  • [–] 8 points 7 months ago

    With how massive of a computer science field artificial intelligence is and how much of it already is or is getting added to every piece of software that exists, a label like that would be equally useless as the California prop 65 cancer warnings.

    Do you use a mobile keyboard that supports swipe typing and has autocorrect? Remember to mark everything you write as being AI assisted.

  • source
  • parent
  • [+] -9 points 7 months ago (8 children)
  • [–] 17 points 7 months ago (7 children)

    If it was the law then the AI itself would be coded to not allow going "undercover", and there would be legal consequences if caught. Torvald's stance only matters for how things 'are' not how they 'could be'.

    Would it be a cure all? Of course not. Fraud still happens despite the illegality. But it's better than not being able to trust anything ever again.

  • source
  • parent
  • hideshow 7 child comments
  • [–] 12 points 7 months ago (4 children)

    I hate to break it to you, but we're never going to be able to trust anything ever again. At least, not the way we used to. In the future, without any doubt, we are going to need to develop a different model of learning, using, and processing information that considers the provenance of where the information came from and how it got there from essentially first principles. We will have to build a web of investigation and trust to determine and mark what information is trustworthy and what is not, especially new information. None of this exists in any meaningful way yet, and the systems we used to have for it, like academic research and journalism for example, would have been catastrophically inadequate to handle this onslaught even at their peak, and they are nowhere near their peak anymore, having been deliberately eroded into a shadow of their former effectiveness so some assholes could get rich and powerful. So hopefully we'll be able to rely on solid ground like Wikipedia and... books as a starting point, and nobody gets around to burning the Library of Alexandria down in their rage against "woke stuff", because otherwise we're going to be rebuilding our information spaces pretty much from scratch in the near future, probably at the same time we're rebuilding civilized society in general. If this sounds incredibly uncertain, tedious and painful: yes, it will be, especially at first. But we will get better at it, eventually. We will develop new systems for it, we will become fluent in information again and the friction will fade.

    I wish we could get to that stage right away, but unfortunately it will have to wait. We can't do anything to improve the swimming pool while we are currently drowning in it. This is the reality that rampant and unchecked use of AI technologies by soulless corporations and corrupt governments have wrought. Logic and reason never stood a chance, and we are entering the digital dark ages. The enlightenment is probably coming someday, but don't hold your breath for it.

    Support your local library, that's the most helpful thing I can think of for individuals to do. Librarians know their shit.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 8 points 7 months ago (1 child)

    we are going to need to develop a different model of learning, using, and processing information that considers the provenance of where the information came from and how it got there

    They used to teach this in schools under "critical thinking skills". Following the chain of sources to the primary sources was a task I had to to (at least in part) more than once in secondary school.

    Authoritarians don't like that tho.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 2 points 7 months ago

    Absolutely, just like addiction to fast food causes obesity, our addiction to fast information has developed into a profound societal ignorance. Studying issues seriously takes time and effort, and if you think "ain't nobody got time for that" I'll tell you right now you're going to have to start to make time for it. Because if you don't, you'll end up knowing nothing, and being wrong about everything, and while that may be acceptable to anyone following all the other lemmings in the same direction (the double irony of "lemming behavior" being historical fake information itself, while posting this on lemmy is not lost on me), I'm also going to suggest to you there will be serious personal consequences from being wrong all the time, and those consequences are going to catch up with you sooner or later.

  • source
  • parent
  • [–] 3 points 7 months ago

    I agree. I've thought a lot about how valuable signing a simple message with a key can be. In an age where machines can appropriate your likeness, how do you accumulate and shed reputation, how do you prove it was you? One low tech version was taking a photo with a newspaper to prove you are a real person. Another is exchanging a public key with a person in real life so you can have reasonable certainty that communications signed with that key are legit. Since this boils down to denying what our eyes have seen, governments and businesses who are very keen on control reality are making their plays. Even identifying yourself cryptographically is only a temporary fix to maintain an existing identity. Your kids will be profiled and mimicked from day one. This whole slippery slope we've been sliding down lately seems very foreseen. It feels like these traps were engineered a very long time ago.

    I think education is the absolute most important thing for a functioning post-truth society. Kids need to smell shit from 20 miles away because the world is full of traps for your mind same as it is for your wallet and your physical body. We also need to be able to verify and trust our tech stack. We need to pass down the stories of the times common people lost and the times common people won. We need to read and discuss philosophy. We'll also have to tackle American religion head on. Also excessively addictive entertainment designs. We are a deeply flawed society and I'm not sure where we should start except for taking some of our time back so people actually have the opportunity to think about these things.

  • source
  • parent
  • [–] 1 point 7 months ago

    If you can spare the disk space, save a local copy of Wikipedia, save pdfs of your favorite books, textbooks, etc., project gutenberg, kiwix library, git mirrors, archive.org, jstor, etc.

    The more people who have their own copies, the better this stuff has a chance of surviving the dark ages.

    Also, if people can figure a way to send/receive data and remotely access servers over mesh networks, it will help populate the new web with useful information. Keep the light alive, even if it doesn't reach everybody. Even through the dark ages in history, knowledge was preserved in monasteries.

    Lastly, although you probably can't grab every news article ever written, be sure to save the ones that are especially salient from a few reliable sources. Future historians and digital archaeologists will thank you.

  • source
  • parent
  • [–] -5 points 7 months ago

    and there would be legal consequences if caught.

    Like for driving over the speed limit? Or putting glass in the regular trash instead of the recycling? Yeah, just what I need in my life, another arbitrary law that's enforced 0.0001% of the time as a flex by the people in power to target and abuse people they don't like.

  • source
  • parent
  • [–] 84 points 7 months ago (7 children)

    Be careful not to introduce security vulnerabilities such as command injection, XSS, SQL injection, and other OWASP top 10 vulnerabilities. If you notice that you wrote insecure code, immediately fix it.

    Lmao. I'm sure that will solve the problem of it writing insecure slop code.

  • source
  • hideshow 7 child comments
  • [–] 34 points 7 months ago (4 children)

    It doesn't fix it, but as stupid as it looks, it should actually improve the chances.
    If you've seen how the reasoning works, they basically spit out some garbage, then read it again and think whether it's garbage enough or not.
    They do try to 'correct their errors', so to say.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 11 points 7 months ago (2 children)

    That’s not enabled by default afaik and it burns through way more tokens looping its output through several times. It also adds a bunch more context which will bring you that much closer to context collapse.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 2 points 7 months ago

    It will slightly improve the chances. But, is that enough?

    Imagine you had an intern working with you on a project. They didn't know anything about SQL injection, cross site scripting, etc. You probably wouldn't give them a task where that was a concern. If you did, you'd watch them like a hawk. Because they're an intern, the amount of code they'd produce would probably be pretty low, and it would be pretty low-quality overall, so it would be easy to spot mistakes that would lead to these kinds of vulnerabilities.

    An LLM has the understanding of the problem space that an intern does, but produces vast amounts of code extremely quickly. That code is designed to "blend in", i.e. it's specifically trained to look like good code, whether it is or not. Because of "vibe coding", people trust it to do all kinds of things, including implement bits where there's a danger of XSS or SQL injection. And the way Claude Code ensures it doesn't generate those vulnerabilities is... someone says "hey, don't do that, ok?"

    Having that statement in there is better than not having it. But, it's just a reminder that these things aren't appropriate for writing production code. They don't actually understand what XSS or SQL injection are, and they can't learn. They don't know why it's important. They don't have a technique for checking if their code actually has those vulnerabilities, other than passing it to themselves recursively and asking that other version of themselves to generate some text that might flag if those vulnerabilities were spotted. But, AIs are famously sycophantic so even recursively using itself, it will generate text to "please" itself and probably write something like "your code is great and I can't spot any vulnerabilities at all! Congratulations! [Emoji] [Emoji] [Emoji]"

  • source
  • parent
  • [–] 68 points 7 months ago (2 children)

    Normally, I’d be reading about NPM security breaches and AI security breaches separately, but now I can get them in the same article! Truly amazing how technology has progressed.

  • source
  • hideshow 2 child comments
  • [–] 115 points 7 months ago* (last edited 7 months ago) (4 children)

    If you installed or updated Claude Code via npm on March 31, 2026, between 00:21 and 03:29 UTC, you may have inadvertently pulled in a malicious version of axios (1.14.1 or 0.30.4) that contains a Remote Access Trojan (RAT). You should immediately search your project lockfiles (package-lock.json, yarn.lock, or bun.lockb) for these specific versions or the dependency plain-crypto-js. If found, treat the host machine as fully compromised, rotate all secrets, and perform a clean OS reinstallation.

    Lol 😂

  • source
  • hideshow 4 child comments
  • [–] 18 points 7 months ago (2 children)

    This is because if an unrelated hack on npm’s latest build. Anyone with this version of npm is affected

  • source
  • parent
  • hideshow 2 child comments
  • [–] 66 points 7 months ago
    [–] 19 points 7 months ago (1 child)

    In this mode, the agent performs "memory consolidation" while the user is idle. The autoDream logic merges disparate observations, removes logical contradictions, and converts vague insights into absolute facts.

    this blog post reads like a marketing piece

  • source
  • hideshow 1 child comment
  • [–] 18 points 7 months ago
    [–] 17 points 7 months ago

    At its core is MEMORY.md, a lightweight index of pointers (~150 characters per line) that is perpetually loaded into the context. This index does not store data; it stores locations.

    Actual project knowledge is distributed across "topic files" fetched on-demand, while raw transcripts are never fully read back into the context, but merely "grep’d" for specific identifiers.

    This "Strict Write Discipline"—where the agent must update its index only after a successful file write—prevents the model from polluting its context with failed attempts.

    For competitors, the "blueprint" is clear: build a skeptical memory. The code confirms that Anthropic’s agents are instructed to treat their own memory as a "hint," requiring the model to verify facts against the actual codebase before proceeding.

    Interesting to see if continue.dev takes advantage of this methodology. My only complaint has been context with it.

  • source
  • [–] 15 points 7 months ago

    I was like "Ha, ha nice April's fools"... Then I keep reading the comments and... WTF‽

  • source
  • [–] 14 points 7 months ago (4 children)

    Best part of the leak, they use regex matches for sentiment lol

  • source
  • hideshow 4 child comments
  • [–] 8 points 7 months ago (3 children)

    I think saw one of the keywords was dumbass. And another looked for you calling it a piece of shit

  • source
  • parent
  • hideshow 3 child comments
  • [–] 20 points 7 months ago (4 children)

    By 4:23 am ET, Chaofan Shou (@Fried_rice), an intern at Solayer Labs, broadcasted the discovery on X (formerly Twitter).

    Ha, by an intern

  • source
  • hideshow 4 child comments
  • [+] -8 points 7 months ago (2 children)

    Against best practice of informing the company first to remediate. Now it’s a security nightmare for anyone running it locally

  • source
  • parent
  • hideshow 2 child comments
  • [–] 11 points 7 months ago

    Once companies started suing people trying to practice "responsible disclosure", I stopped attacking people that choose maximum disclosure.

    Responsible disclosure has always been a bit of a hedge. It's rare to be able to show you are actually the first person/organization to discover a vulnerability.

  • source
  • parent
  • [–] 9 points 7 months ago

    The best learning method is from your own mistakes. So, Claude is still learning.

  • source
  • [–] 21 points 7 months ago* (5 children)

    Like a healthy brain. And just like a healthy brain, it'll still hallucinate and make mistakes probably:

    The leaked source reveals a sophisticated, three-layer memory architecture that moves away from traditional "store-everything" retrieval.

    As analyzed by developers like @himanshustwts, the architecture utilizes a "Self-Healing Memory" system.

  • source
  • hideshow 5 child comments
  • [–] 18 points 7 months ago (4 children)

    We’re gonna make AGI and realize that being stupid sometimes and making mistakes is integral to general intelligence.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 29 points 7 months ago* (last edited 7 months ago) (5 children)

    I mean it's not that big a deal. However, it would another thing if the model itself leaked. Now that would be something.

    edit: Like I thought, it turns out to be a TS wrapper with more internal prompts. The fireship video is really funny, they use regex to detect if the user is angry 😭

  • source
  • hideshow 5 child comments
  • [–] 2 points 7 months ago (2 children)

    Tool usage is very important. Qwen3.5 (135b) can already do wonderful things on OpenCode.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 14 points 7 months ago* (1 child)

    I dabble in local AI and this always blows my mind. How do people just casually throw 135b parameter models around? Are people like, renting datacenter hardware or GPU time or something, or are people just building personal AI servers with 6 5090s in them, or are they quantizing them down to 0.025 bits or what? what's the secret? how does this work? am I missing something? like the Q4 of Qwen3.5 122B is between 60-80GB just for the model alone. That's 3x 5090s minimum, unless I'm doing the math wrong, and then you need to fit the huge context windows these things have in there too. I don't get it.

    Meanwhile I'm over here nearly burning my house down trying to get my poor consumer cards to run glm-4.7-flash.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 3 points 7 months ago

    This is just the UI right? Or the models too?

  • source
  • [–] 3 points 7 months ago

    Vote people. There's town and city votes everyday or often. Vote!

  • source
  • [–] 1 point 7 months ago

    Actual project knowledge is distributed across "topic files" fetched on-demand, while raw transcripts are never fully read back into the context, but merely "grep’d" for specific identifiers.

    Consistent with a lot of bugs and goofs I've heard people in long running instance of Claude will encounter.

  • source
  • 1 comment in c/artificial_intel@lemmy.ml view that post
    [–] 2 points 7 months ago

    It sounds like this is a big deal. I thought most AI models were built this way already though, as operating systems for LLM prompt generation. This seems like an iteration of that that includes an ability to store certain data it's referring to frequently as "facts", and let other data degrade over time to allow it to learn new things. Is that accurate?

  • source