221
AI staff 'genuinely frightened' for humanity's future, ex-Anthropic researcher tells BBC (www.bbc.com)
submitted 1 month ago by to c/technology@lemmy.world
213 comments fedilink hide all child comments
 

I'm not an AI techie and I'm not well educated on LLMs/AI in general. The issue I'm having is the dichotomy between how people talk about it. I've heard completely opposite ends of the spectrum. From AI is quickly becoming advanced enough to threaten human society, all the way to its a glorified dictionary/search engine that can't really do much of anything. I know Lemmy has a steep anti-AI slant. But, can someone talk about this in a way I can wrap my head around why the divide seems so big?

you are viewing a single comment's thread
view the rest of the comments
[–] 23 points 1 month ago (22 children)

I've heard a few explanations for the "AI will kill us all" narrative.

  1. AI techies are in a cult. They genuinely believe this stuff because they're crazy.

  2. It makes the AI sound powerful and impressive, which is ultimately good for the AI companies.

  3. They're starting to face competition from open-weight models, so they want to scare politicians into crafting regulations. They think that they will be able to influence the regulations to favor them and squeeze out their competition.

And, of course, there's

  1. It keeps people talking about AI. Because whatever you do, don't quit talking about AI, because if you quit talking about it, the line might go down! And the line going down is very, very bad.
  • permalink
  • fedilink
  • source
  • hideshow 22 child comments
    •
  • [–] 3 points 1 month ago

    I am in a cult, the cult of the machine god. Use your imagination for a second, trillions of dollars spent on this technology for A: Greed and random speculation B: Rich people really want this to control you better. C: a handful of tech CEOs are running the worlds largest grift. D: The machine god was awoken years ago and we're building data centers at its behest.

  • permalink
  • fedilink
  • source
  • parent
  • [–] [S] 3 points 1 month ago (18 children)

    Okay cool, thanks!

    1. If the techies believe it and they are the "experts" what makes them crazy? Wouldn't their opinion be the most informed?

    2. Yeah that part definitely makes sense

    3. That definitely makes sense from a capitalist perspective. But, is the argument against regulations that they would be bad for the concentration of power it would give select companies? Or just that they aren't necessary because the tool is of little use so it's just wasted time/money.

    4. That makes sense but trillions of dollars are being spent. Anything that amount of money is sinking into is gonna be talked about a lot. Especially thousands of jobs cut because of it.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 18 child comments
    •
  • [–] 4 points 1 month ago (2 children)

    I think there is something more interesting going on in the Jacob Coxon story because it's [the] phrase "everyone at anthropic believes that AI is an existential risk"... Every organization has to instill some kind of beliefs among its members in order to do the work it does, right? Everybody at Goldman Sachs believes the free market is great, capitalism is the best system, and that they are the wealth creators who deserve big bonuses. Everyone in the Mormon church believes they should all tithe 10% of their income, and that they each get a special planet when they die. And everyone who worked at Philip Morris in the 80s believed that cigarettes tasted great and did not cause any of that nasty cancer. There's Upton Sinclair quote, "It's difficult to get a man to understand something when his salary depends on his not understanding it." Well, the opposite is also true. It's guaranteed that someone will believe something if their salary depends on them believing it.

    Now, let's just grant that all the people in these organizations definitely know things about finance and Mormonism and cigarettes and AI that you might not know because they are insiders and you are not. They have access to information and experiences that you do not have. And that might even make it a little bit difficult for you to win an argument with them. But arguing with their belief -- trying to figure out whether or not its truth value is true or false -- is not the only way to take a skeptical attitude towards it. We can also ask, what work is the belief doing for them. What does the thing that they believe allow the organization they're a part of to do that it wouldn't be able to do if the people in the organization didn't believe that thing?

    And this started to become clearer to me when I started listening to the actual claims that Jacob Coxin was making because a lot of them don't really add up.

    https://www.youtube.com/watch?v=pauBYSQ1K_c

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 1 month ago (1 child)

    There's a huge difference between this "AI scare" and everything else in your list. A whistleblower declares "the corporate culture is insane and will kill us all." This time the corporate leaders reply "that's exactly right". Just because you are right to not trust the bond villains in how they suggest they should be regulated, does not mean you should dismiss the underlying problem these experts+ especially the expert leaders would prefer to take all your money to accelerate the destruction instead of have society worried about the consequences of giving them all your money.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 1 month ago

    There are plenty of underlying problems to discuss. Adam Conover mentions chatbots encouraging suicide and murder. Guess who doesn't: the "experts and expert leaders." The ones who are happy to repeat the rhetoric that is either untrue, highly theoretical, or conveniently unproven.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 3 points 1 month ago* (last edited 1 month ago) (6 children)

    If the techies believe it and they are the “experts” what makes them crazy? Wouldn’t their opinion be the most informed?

    Listening to them speak about it, no. They're morons. They clearly didn't make the tech. Workers made AI. Just like they make everything. These freaks make nothing. They can't give single coherent explanation of how LLMs work, how they could "kill" us, nor do they seem to know how or what "guardrails" need to be put in place.

    But if they really are a cult then we all know what the next step is. They need to STFU and get off of our planet.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] [S] 4 points 1 month ago (5 children)

    That's what I mean though, the subject of this article was a 27 year old worker. Not some high level Sam Altman type.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 5 child comments
    •
  • [–] 0 points 1 month ago (4 children)

    They aren't a homogenous group. They don't all believe this. The "AI will kill us all" narrative just sells really well.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 1 point 1 month ago (2 children)

    Right im sure the people working adjacent to the DoD have more insight and fears than the people working with the consumer facing models.... Again what motive does a young kid have for quitting the company and going on a media tour?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 1 month ago (1 child)

    A Google employee claimed in 2022 that they had sentient AI. He was sacked. What were his motivations? Do you believe he was right?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 2 points 1 month ago

    Absolutely do no believe he had a sentient AI back then or now. I'm not talking about a thing with feelings. They don't need sentience to pursue and complete objectives, they dont need sentience to self-preserve or self improve. I'm more concerned about a rogue instance that's just doing what it was told to do autonomously. Perhaps upgrading itself while it attempts to gain access to launch codes. I'm concerned that a well financed terrorist org can spin up bioweapons. Like I am not taking the AI labs at their words, I am speaking from my experience as a explorer of the technology. I'm thinking about the path from ChatGPT to the newest model, and how they will look in 2 years when more data centers are online and the limitations on training and teaching slowly disappear.

  • permalink
  • fedilink
  • source
  • parent
  • [+] 2 points 1 month ago (6 children)
    [deleted]
  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 3 points 1 month ago (4 children)

    We're talking about mid-tier employees that left the company.... I mean i guess there incentives could be to pump their shares they undoubtably have... They're all quite wealthy now in context. Maybe he just wanted 5 minutes of fame...or maybe they understand the trajectory of the technology. Maybe they are genuinely concerned and we should at least acknowledge that?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 0 points 1 month ago (3 children)

    There are motives for whistleblowers to lie. Book sales, political careers, other attention. The executives they accused, though, just said they were right

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 2 points 1 month ago (2 children)

    of course there's motivation for everything, which one are you assigning this individual and based on what technological disagreement?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 1 month ago (1 child)

    when the boss who is being whistleblowed against admits the whistleblower is right, you can believe both that there are problems. I can't think of any other case in history where this happened. The disgruntled ex-employee smear is a common defensive move instead.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 2 points 1 month ago

    Thats sound logic, i can't disagree with that. I still think having a machine that can automate scripting of malware is pretty dangerous to the digital society we set up. If the grid fails and the financial system collapses, I'm gonna be okay... Are you?

  • permalink
  • fedilink
  • source
  • parent
  • [–] 0 points 1 month ago
    1. They’re in a cult that was founded by someone who has no expertise or understanding of the technology.

    He watched a bunch of sci-fi movies and got shook because he’s a god damn moron. He has since started researching things, but it is all under the guidance of his ungrounded beliefs. Confirmation bias.

    Their “movement” is literally based off of a Harry Potter fan fiction novel.

    Taking them seriously is equivalent to taking the Scientologists seriously. Do you think the Scientologists know the most about space just because they spend the most time reading stories about it?

    https://en.wikipedia.org/wiki/Rationalist_community

    https://en.wikipedia.org/wiki/Harry_Potter_and_the_Methods_of_Rationality

    These are not serious people

  • permalink
  • fedilink
  • source
  • parent
  • [–] 1 point 1 month ago

    as soon as we figure out how to break the laws of thermodynamics and invent infinite energy, we can feed that to ai to finally gain real ai and it will solve perpetual motion and faster than light travel. get ready because once all this happens and the stars align, the fountain of youth will rise in the bermuda triangle and the ufos will bring back jesus(white and original flavor). this is going to unlock all the mysteries of the universe.

  • permalink
  • fedilink
  • source
  • parent