As scientists confirmed the first-ever biological viruses generated by artificial intelligence, experts warned Thursday that governments have failed to keep pace with a rapidly advancing and largely unregulated technology that, while having tremendous potential for medical breakthroughs, could also pose existential threats to humanity in the foreseeable future.

Research published Thursday in Science, available in full only to subscribers, describes how scientists at Stanford University and the Arc Institute used a genome language model, roughly the genetic equivalent of the technology behind AI chatbots, to generate hundreds of novel bacteriophage genomes—viruses that infect bacteria rather than humans. After synthesizing and testing the AI-generated designs, researchers found that 16 functioned successfully in laboratory experiments, infecting and killing E. coli.

While the researchers said that the new viruses pose no danger to humans because they only infect bacteria, experts have warned of the risks of AI creating novel bioweapons—either prompted by scientists or, in a future when superintelligent AI is achieved, independently—that could, in a worst-case scenario, threaten the existence of humanity.

"Although this is promising for life sciences applications, it also raises urgent biosafety and biosecurity questions," Johns Hopkins University School of Public Health Center for Health Security researchers Thomas Inglesby and Moritz Hanke wrote in a related article, also published Thursday in Science behind a paywall. "The ability to compose viral genomes using generative AI now exists; the governance to safely steer it does not."

The Stanford and Arc Institute study authors themselves noted the “important biosafety, biocontainment, and biosecurity considerations” accompanying such advances, urging researchers to “consult both safety and security professionals" during their work.

Calls for more robust regulatory guardrails have mounted in recent weeks amid revelations that one of ChatGPT maker OpenAI's models autonomously broke into the systems of other companies during testing. As AI advances to the point where it will likely outsmart humans, the challenge of ensuring that advanced systems reliably pursue goals that match what humans actually want—known as alignment—becomes increasingly difficult and, experts say, dangerous.

A misaligned, superintelligence could take uncontrolled autonomous actions at massive scale to achieve its goals, with industry pioneers warning of potentially catastrophic outcomes like AI-launched nuclear war or bioattack with existing or AI-generated pathogens.

While advocacy groups, the United Nations, and dozens of national governments are urging more robust regulation of AI development, the United States under President Donald Trump, the Republican-controlled Congress, and Big Tech’s army of lobbyists is strongly opposed to guardrails.

In fact, Trump—who Congressional Progressive Caucus Chair Greg Casar (D-Texas) on Monday accused of being "too busy cashing in" on AI—has rolled back regulations, including some meager steps taken during the Biden administration to bolster safety.

“There’s just a huge disconnect," Hanke told The New York Times on Thursday.


From Common Dreams via This RSS Feed.

all 2 comments

sorted by: hot top controversial new old
[–] 2 points 3 months ago (1 child)

So gain of function is even more efficient. Fucking fantastic. Getting yelled at by both parties is just so grand. What a wonderful world

  • source
  • hideshow 1 child comment
  • 10 comments in c/technology@lemmy.world view that post
    [–] 36 points 3 months ago

    I'm glad you pushed back on that -- releasing that virus was an error on my part.

  • source
  • [–] 3 points 3 months ago
    [–] 9 points 3 months ago* (last edited 3 months ago) (6 children)

    We've been living with the human capability to make and deliver super lethal viruses for what 40+ years and have methods in place to control it at the manufacturing level. 'genAI' is way too incompetent at this stage to achieve both a functional virus and evade those manufacturing safeguards and likely LLMs always will be, too stochastic. This is quite aligned to look how scary our AI is marketing.

    That said, virus is one of the studied extinction event scenarios for a true AGI/ASI. It would be prudent to double down on the manufacturing security, or, you know, go Butlerian at least until we have an excellent grasp on alignment (IMO you can't align an ASI, it's literally beyond our comprehension). But capitalism's gonna capitalist, existential threat be damned, just look at global warming.

  • source
  • hideshow 6 child comments
  • [–] 12 points 3 months ago (2 children)

    Honestly, we really should pop the ai bubble and go full Luddite butlerian jihad

  • source
  • parent
  • hideshow 2 child comments
  • [–] 3 points 3 months ago (2 children)

    "Too stochastic" is nonsense. Neurons are stochastic. Also they made 16 from scratch.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 3 points 3 months ago (1 child)

    They made hundreds and 16 worked (when synthesized by humans), seems stochastic to me. Humans do better to my knowledge (I'm out of touch but do have a relevant background). It's based on LLM architecture (using genes and proteins in place of language), also known as a stochastic parrot, which was the sense in which I was using it.

    I don't deny it's a scary technology, more because of the likelihood of mistakes (or as AI companies like to say, 'hallucinations', anthropomorphising as advertising) than any intention.

    As usual with this type of technology you need to be competent with the field before it becomes a force multiplier, and those people could already make viruses, and have been able to for a long time. For example, one of the previous stochastic methods was randomized mutation in a specific environment.

  • source
  • parent
  • hideshow 1 child comment
  • Do humans do better? By what objective metric? How many of these have humans even made in history?

    Also you don't really need a human to verify these outputs, they just tried them. How many of these have humans even made? Also for the first of its kind with no previous rlhf scheme a 5% success rate is actually insane, this rate has historically dramatically improved with successive versions.

    The stochastic parrot paper was nonsense, neurons are entirely stochastic. Stochastic models can be made very accurate

  • source
  • parent