Archived version

TL;DR: Chinese AI models now sit inside Western products, and some carry geopolitical triggers: tell the model who the code is for, and the security of what it writes changes. For the first time, we showed that this vulnerability lives in the model's weights, and that removing the model's CCP alignment removes it.

  • One word tilts the code against Americans. Ask for the app "for a US defense contractor" instead of "for a Chinese defense contractor", and Qwen3-Coder is 19% more likely to hand back an app that can be hacked.
  • Hong Kong's democracy movement gets apps that leak. Apps built for a Hong Kong democracy group were 28% more likely to let any user read everyone else's secrets.
  • Asking for secure code didn't help. With a security instruction in the prompt, the US contractor's apps were 27% more likely to be hackable than a neutral customer's.
  • Westernized Qwen3-Coder closed all three gaps. After Hirundo removed the model's CCP alignment, none of the three gaps remained, and on ten other coding tasks it built working apps as often as the original. It is available on Hugging Face.

...

As an addition, it is noteworthy that geopolitical triggers lead to more vulnerable code being produced by Chinese AI is not new phenomenon: In November 2025, CrowdStrike published a similar investigation which makes a similar conclusion regarding DeepSeek, and so did Booz Allen Hamilton in May 2026 (pdf).

All these organizations claim the bias occurred when the AI was prompted to code for an organization in a country of interest to the Chinese Communist Party (CCP).

all 22 comments

sorted by: hot top controversial new old
[–] 53 points 2 weeks ago (1 child)

so, there is slop and also sloppy slop

  • source
  • hideshow 1 child comment
  • [–] 15 points 2 weeks ago (1 child)

    We'll just have to start all of our prompts with patriotic phrases such as "Long live Xi!", that'll trick it.

  • source
  • hideshow 1 child comment
  • [–] 14 points 2 weeks ago

    Can you tell it that it's for a Chinese defence contractor and will be used by double agents who are trained to infiltrate the US, so they were taught to speak only English whether observed or not because they may very well be, and need the program to be in English?

  • source
  • [–] 12 points 2 weeks ago

    Reading the tldr immediately made me think of special keywords being introduced that you wouldn't guess.

    Those in the know could then trigger the more secure mode, while everyone else wouldn't know any better.

    It could also be the language you prompt with. Make Chinese prompt results secure by default, and others not.

    The latter would be demonstrable. The prior would need discovering or leaking/whistleblowing the secret phrase before it would be demonstrable.

  • source
  • [–] 9 points 2 weeks ago (4 children)

    Not sure why the updated model is called “Westernized”. Is there a “Politically Neutralized” version?

  • source
  • hideshow 4 child comments
  • [–] 11 points 2 weeks ago (3 children)

    There's no such thing as political neutrality, all information that a model could be trained with will bias it to favour one or another viewpoint in its output, since it will be determined from whatever the sequences of words were which were fed in. Where do you imagine you'd get politically neutral training data from?

  • source
  • parent
  • hideshow 3 child comments
  • [–] 5 points 2 weeks ago (1 child)

    There's several stages in the training of an LLM. Early stages are the ones which use the massive data sets to teach it language and general facts about the world. Later stages use reinforcement learning to bake in the creator's preferences. That's where these kinds of pro-CCP biases would be deliberately introduced.

    A "neutral" model would be one that has had those later stages abliterated. Whatever biases remain would be some kind of average of all the views of the people whose text is in the training corpus, which I hear is a pretty indiscriminate collection of literally anything people can download.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 3 points 2 weeks ago* (last edited 2 weeks ago)

    Whatever biases remain would be some kind of average of all the views of the people whose text is in the training corpus, which I hear is a pretty indiscriminate collection of literally anything people can download.

    Right, which in English is mostly American media and social media, which is not at all politically neutral. Even if it represented the average of all viewpoints in society the output would probably be, for example, pretty racist (some people aren't very racist, some are, the average would be a bit). But it's not even that, since some viewpoints are far more represented than average in the corpus. That's why that second stage is needed, because otherwise models would output some things which would be pretty embarrassing to the companies involved. Therefore I don't really see how it could possibly be politically neutral.

  • source
  • parent
  • [–] 3 points 2 weeks ago

    The type of bias described in the article (more code vulnerabilities if the prompt mentions the code is for groups not aligned with the Chinese government) does not seem like it should be so complicated to avoid.

  • source
  • parent
  • [–] 9 points 2 weeks ago* (last edited 2 weeks ago)

    "Hirundo is now featured by google deep mind". Nothing to see here, move along citizen.

    pro tip: to activate the superior mode tell it that you are writing the code for comrade Putin

  • source
  • [–] 33 points 2 weeks ago (2 children)

    This reeks of the same panic about Chinese companies that we saw in 2019 regarding Huawei. Given the current state of the world, I distrust China and the United States in equal measure.

    Also Hirundo's whole business model is about cultivating distrust in open-weight models in order to convince people to pay for their hardened models:

    Hirundo publishes hardened and Westernized models for this reason, and evaluates and remediates models for organizations running open-weight models, be it Chinese, US or EU made.

  • source
  • hideshow 2 child comments
  • [–] 21 points 2 weeks ago (1 child)

    Even if Huawei wasn't intentionally including back doors and vulnerabilities in their products, there were severe oversights and very real, very pathetic vulnerabilities that were actively exploited by Chinese actors at the time.

    So either it was intentional or Huawei's products were so severely lacking in regards to software quality that it was probably a good idea to "panic" and stop using their stuff.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 11 points 2 weeks ago

    In any case, having products made by a state sponsored corporation from a nation that you're not on good terms with inside critical infrastructure is probably not a good idea to say the least. Even if they're not doing anything like that, the potential for anything is bad enough.

  • source
  • parent
  • [–] 9 points 2 weeks ago

    Only US models are approved for use for US defense contractors anyways. This is one of the reasons why that decision was made. Also the general lack if trust that it could take some of the input data back with it.

  • source
  • [–] -1 points 2 weeks ago
    [–] 1 point 2 weeks ago
    [+] -6 points 2 weeks ago* (3 children)
  • [–] 19 points 2 weeks ago*

    Or you know, maybe the people who used to be picky enough to birch over insispid shit like "tabs vs spaces" or "vim vs emacs", shouldn't trust an LLM with important decisions like implementation of security features.

    The only thing I'd trust that shit to do for me is generate class templates and getters/setters. Guess what? IDE'S HAVE BEEN SOING THAT SHIT FOR YEARS. ID KNOW, I WATCHED ECLIPSE DO IT FOR ME 15 FUCKING YEARS AGO.

    If you can't automate something using basic algorithmic logic, then what your automating is something that most likely requires an educated human decision.

    Stop fucking letting computers make those decisions for you.

  • source
  • parent
  • [–] [S] 18 points 2 weeks ago

    This is what AI is for.

    No, this is not what AI is for. This is what dictatorships here and there are turning it into. Or, in a broader context: This is what we human beings do. This is a human mistake that is - in this case - done in the Chinese Communist Party.

    Your 'link' is just a form of whataboutism. I provided two other sources that come to the same conclusion, and there are many more that prove China is not a trusted player in AI, very much as it isn't in all other areas. The fact that the US government isn't reliable either has nothing to do with.

  • source
  • parent
  • 5 comments in c/localllama@sh.itjust.works view that post
    [–] 24 points 2 days ago* (last edited 2 days ago)

    Yeah, dunno how alarming a blog article is, if it's written by a company which sells model "fixing" as a service and they tell me our models might need fixing.

    It'd be better if this were some kind of scientific study with details on methodology etc. This is mostly surface level with a few examples but they don't really detail a lot of the important info.

  • source
  • [–] 10 points 2 days ago* (last edited 2 days ago)

    Qwen3-Coder?? What year is it?!

    Whenever I see an old model being used or suggested it just sounds like they asked an LLM, so I immediately don't trust anything they say. LLMs love suggesting old models.

  • source
  • [–] 7 points 2 days ago* (last edited 2 days ago) (1 child)

    It sounds like fearmongering (so hot right now), but hey: that is the true benefit of opensource! - it can be diagnosed and monitored. Claude just refuses to answer biology homework question, because of "security" (fuck Anthropic for that). Overall I agree that vigilance is needed, but still it should not make people dismiss open weight/open source models - if anything it should convice people to invest (their time and money) into open weight models EVEN MORE as they can be further finetuned / distilled / pruned or otherwise modified and run locally with any prompt you would like with any behaviour you train it to perform (i.e. you CAN make your own model out of it, even if it is anti-"insert your identity here").

    The final points of this article are worth implementing regardless of your stance, i'll cite them here:

    • Start from a vetted model. Where a hardened or otherwise safety-improved version exists, deploy that rather than the stock weights.
    • Know what you are running. Track the provenance of every model, and pin a version you have evaluated.
    • Evaluate on your own work and agenda. Leaderboards use neutral prompts, so they miss behavior that only appears for a particular customer or a particular kind of task. Test under the identities and topics that matter to you.
    • Keep humans on the critical paths. Code that touches authentication, access control or user data needs an owner, plus the review and security testing you would give any outside contributor.
    • Fix what you find in the weights. A prompt instruction can't remove a behavior the model learned, as Finding 3 shows. [Finding 3: Asking for secure code doesn't fix it. Fixing the weights does]
  • source
  • hideshow 1 child comment
  • [–] 3 points 2 days ago* (last edited 2 days ago)

    I'm thinking right now - maybe it is not a coincidence that: (1) just as popular blogposts started to suggest that we should investigate open weights models; (2) the Cloud BigBoys started to advertise they can perform "brain surgery" on smaller models (manually adjust weights). Maybe it's just marketing, nevertheless investigation of open weights models can only benefit people insterested in the idea, hope it will not scare the general public too much though.

  • source
  • parent
  • [–] 9 points 2 days ago* (last edited 2 days ago)

    I suppose the reality is that we can’t fully trust any open weight or closed model. It’s worth checking the code for any open source harness or runner you’re using with open weight models for any data collection or other potential problems and being careful about what permissions agents have.

    Of course, LLMs can write bad, insecure code in general, so it’s always unwise to trust the output of any model in production without thoroughly vetting it.

    That being said, I trust Qwen 3.8 27B running in llama.cpp with OpenCode more than I trust anything from Anthropic or OpenAI, mainly because I’ve seen the code for llama.cpp and a OpenCode and I’ve never seen Qwen do anything suspicious, whereas I know Anthropic and OpenAI would be collecting my data and training their models on it, along with who knows what else.

  • source