Apparently AI under the control of a rogue actor is undoubtedly going to kill us all with a synthesized virus.

Let me run you through how brain rotted the world is now that this is even being taken in the slightest seriousness.

You cant just prompt an AI model, hey make me a humanity ending virus, no mistakes and send it to the printers. That's not how reality works. To generate a world ending virus. You will need the following chain of events. If each one fails, the whole chain of events fail.

Step A: AI generates a viable novel pathogen sequence.

Highly unlikely without massive human intervention and iterative wet-lab feedback. Let’s say 1% chance for a non-expert using public tools.

Step B: The sequence is stable enough to be synthesized and remain infectious.

Many generated sequences are nonsense or unstable. Probability of viability: 5%.

Step C: It bypasses DNA synthesis screening (Know-Your-Customer rules).

Screening is imperfect but effective against known dangerous motifs. Probability: 10%.

Step D: The actor has access to a BSL-3/4 lab capable of handling high-consequence pathogens.

You forget extensive of testing of the novel virus canidates is needed so you can validate the virus's virality. You can’t do this in a garage. You need specialized infrastructure, containment, waste disposal, and ethical blind spots. Probability for a random bad actor: <0.1%.

Step E: The virus achieves optimal transmissibility AND lethality simultaneously.

Biologically, these often trade off. High lethality usually kills the host before they spread it widely. Finding the "sweet spot" is incredibly hard. Probability: 1%.

Step F: It spreads globally before detection/response.

Modern surveillance, travel restrictions, and medical response are faster than in 1918 or 2020. Probability of total global saturation before countermeasures: 5%.

Any one of these fail and you dont get an outbreak.

Total Probability:

0.01 ×0.05 ×0.10 ×0.001 ×0.01 ×0.05 =0.00000000000025%

And before you say state actor will do this. They already can if they want to.

all 20 comments

sorted by: hot top controversial new old

Okay, now what are the odds of an AI lab full of hubris removing those safeguards and then giving it the means to synthesize pathogens?

  • source
  • [–] 7 points 2 weeks ago (11 children)

    Unless you have a source for all of those numbers, this is just "I think it's unlikely and here are the things I perceive as obstacles." Adding made-up numbers to them is just noise.

  • source
  • hideshow 11 child comments
  • [–] [S] -2 points 2 weeks ago (9 children)

    lol there is no source bro, its a educated guess. no one has a robust study on viability of some guy setting up a secure wet lab. All im saying is .... its kinda a joke that "AI will kill us all because its going to engineer a deadly virus".

  • source
  • parent
  • hideshow 9 child comments
  • [–] 1 point 2 weeks ago (8 children)
  • [–] [S] -1 points 2 weeks ago* (7 children)

    ok sure i'll bite, either tell me where i can source these numbers or tell me why im not allowed to make an educated guess or what part about these probabilities are you disputing as completely logically unsound. come on, i'll wait for your response. otherwise fuck outta here you big tech shill

  • source
  • parent
  • hideshow 7 child comments
  • [–] 1 point 2 weeks ago (6 children)

    either tell me where i can source these numbers

    That's your problem, not mine. If you don't even know where to look, don't spout off.

    or tell me why im not allowed to make an educated guess

    You're allowed to say whatever you want, and I'm allowed to call it nonsense. I see no reason to call your guesses educated.

    or what part about these probabilities are you disputing as completely logically unsound

    The fact that you're providing literally no basis for them. I could just as easily make your post and change all of the numbers to 100% and I'd have exactly as much evidence as you do.

    otherwise fuck outta here you big tech shill

    I'm not even saying that your conclusion is wrong. I'm saying that your argument is stupid. You might as well have said "LLMs won't eradicate humans with a virus because Mickey Mouse came to me in a dream and told me so."

  • source
  • parent
  • hideshow 6 child comments
  • [–] [S] -1 points 2 weeks ago* (5 children)

    my argument is stupid because ? you still haven't said why ? all you said was ... well i can call you stupid because i can do whatever i want.

    btw can i ask if you have mental retardation ?

  • source
  • parent
  • hideshow 5 child comments
  • [–] 0 points 2 weeks ago (4 children)

    my argument is stupid because ? you still haven't said why ?

    Your argument is stupid because you used a bunch of numbers that you made up. I said that very clearly.

    all you said was ... well i can call you stupid because i can do whatever i want.

    No, that was after I repeatedly pointed out that you just hallucinated the probabilities.

    btw can i ask if you have mental retardation ?

    I assume that you can, yes.

  • source
  • parent
  • hideshow 4 child comments
  • [–] [S] 0 points 2 weeks ago (3 children)

    ohhh ok. so the AI is going to create a virus and kills us all that Dario is preaching. He totally didnt make that one up yeah ?

  • source
  • parent
  • hideshow 3 child comments
  • [–] 0 points 1 week ago (2 children)

    Maybe, maybe not. Given LLMs' success in certain molecular research, like protein folding, I'd guess (see how easily it is to not pretend that making shit up is authoritative?) that the probability isn't zero. I certainly don't trust tech CEOs, though.

  • source
  • parent
  • hideshow 2 child comments
  • [–] [S] 0 points 1 week ago (1 child)

    you sir are an idiot, you just made up non zero probability ROFLMAO mate you are brain rotted. keep on shilling man

  • source
  • parent
  • hideshow 1 child comment
  • [–] 0 points 1 week ago

    And I labeled it as a guess, and my error bars are nearly the entire probability space. I didn't just make up a bunch of numbers and pretend that they were authoritative.

    I literally would not have replied to your initial post at all if you had just listed the obstacles instead of making up numbers and pretending that they meant anything.

  • source
  • parent
  • [–] 3 points 2 weeks ago (2 children)

    @sagemythos11 For a random bad actor this might be true but we live in a world where Nations and billionaires have the resources and the power to do it if they want. Imagine X-"AI" starting a biotech company or the CIA or FSB who already research such pathogens and will/ probably already do use "AI". Would you Trust a Tech Bro, Trump or Putin to do responsible research? I do not!

  • source
  • hideshow 2 child comments
  • [–] [S] 0 points 2 weeks ago (1 child)

    state actors can already do this if they want to. Why does Putin and Trump even need to do this ? They have nuclear codes. A tech bro is the risk vector here.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 2 points 2 weeks ago

    @sagemythos11 Both are. If Putin decides to create more and new pathogens "AI" just accelerates the process and Tech Bros would create a new risk. But none of them should be able to automate this process.

  • source
  • parent
  • [–] 3 points 2 weeks ago

    It’s unlikely that whatever catastrophic outcome occurs will be in anyone’s top 10.

    It’s just as likely some critical component of human survival is tied to AI, AI goes buggy, human survival systems stumble and massive famine results.

  • source
  • [–] 2 points 2 weeks ago

    I've said it before but I feel it's important to note that the tech bros have access to all of the CDC data and Peter Theil is a big investor in AbCellera Biogenics which is a very advanced lab. There is risk in this space.

  • source
  • [–] 1 point 2 weeks ago (1 child)

    It's not some random dude in a garage, it's the unregulated tech overlords and autocrats that could do it. AI isn't intelligent, but it has amazing pattern matching capabilities. It's going to create all sorts of advancements in science, and to deny the problems that comes with this is both silly and dangerous.

    If we want to fight ai, we have to understand it. It's not a conscious omnipotent god like the tech bros think, but it's also not a useless fad. Real scientists are warning of its potential consequences, and governments all over the world are investing trillions to improve it.

    In a decade, it should be at least twice as powerful as it is today, and that is indeed a big deal.

  • source
  • hideshow 1 child comment