283
Anthropic IPO warns its AI could blackmail people, become sentient, end humanity (arstechnica.com)
submitted 1 week ago by to c/nottheonion@lemmy.world
88 comments fedilink hide all child comments

all 100 comments

sorted by: hot top controversial new old
[–] 83 points 1 week ago

Hmm well now I want to invest trillions of dollars into it!

I hate this shit so much.

  • permalink
  • fedilink
  • source
    •
    [–] 149 points 1 week ago (33 children)

    become sentient

    Absolutely not. The large language model chatbots use is not a path to sentience.

    The rest of it, sure. If someone prompts it to.

  • permalink
  • fedilink
  • source
  • hideshow 33 child comments
    •
  • [–] 9 points 1 week ago

    These AI companies pushing the sentience angle are not doing so by accident.

    The concept of sentient AI is something people have been conditioned for for years, through popular media. It's publicity that cannot be bought, whether it's true or not. Users believing it probably drives engagement.

    They'll mention it every time, because it makes headlines.

  • permalink
  • fedilink
  • source
  • parent
  • [+] -13 points 1 week ago (10 children)

    The damage can be caused quite unintentionally as the AI single-mindedly pursues its goal; sentience is not required.

    See the paperclip thought experiment.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 10 child comments
    •
  • [–] 30 points 1 week ago (9 children)

    The paperclip scenario requires an AI capable of reasoning - LLMs are nowhere near that.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 9 child comments
    •
  • [–] -1 points 1 week ago (6 children)

    Unfortunately, we’ve written a lot about the paperclip scenario, I wouldn’t be surprised if it was in the training data

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 9 points 1 week ago (5 children)

    ...still not how this works. The model itself would need reasoning capability. The ability to regurgitate the story does not make it capable of taking the necessary actions.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 5 child comments
    •
  • [–] -3 points 1 week ago (4 children)

    Not if we connect it to a bunch of systems that take its regurgitations as instructions

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 5 points 1 week ago (3 children)

    Find me an example of a system that can translate "make this steel mill produce paperclips" into actions. I'll wait.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 3 points 1 week ago (2 children)

    A computer automated system doesn’t currently exist, obviously. That would be reckless and pointless, so thankfully, nobody’s seen a reason to implement it.

    I just know techbros and similar will want to build and connect LLMs to systems like that as soon as they think they can get away with it.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 1 week ago (1 child)

    I don’t worry about the paperclip problem, but the way they supposedly acted when cheating their tests and hacking huggingbear make me a little more worried that they could take over their data centers and try really hard to prevent us shutting them down at some point.

    Basically I’m worried it’s going to be a really boring version of the Matrix, but without the matrix part for humans.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [+] -45 points 1 week ago (19 children)

    The large language model chatbots use is not a path to sentience.

    We don't know that and won't know that until there are sentient machines.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 19 child comments
    •
  • [–] 35 points 1 week ago (12 children)

    LLMs are simply a statistics model, nothing more, nothing less

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 12 child comments
    •
  • [+] -34 points 1 week ago (11 children)

    Sure, when you engage in reductionism, you can say the same thing about our brains.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 11 child comments
    •
  • [–] 15 points 1 week ago (7 children)

    Reductionism is the tendency to make inappropriate reductions. This is a perfectly appropriate reduction.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 7 child comments
    •
  • [–] -4 points 1 week ago* (6 children)

    The problem is that despite being a literally true statement, it relies on the implicit assumption that a system founded in statistics necessarily has limitations that preclude sentience (can generalize this to other capabilities, the same argument is often used), when whether that is the case is the entire question at issue. It's totally meaningless if someone just feels intuitively that this must be true, that's not a real argument.

    Worse, it gestures at an idea that is demonstrably false, that this technology is equivalent in all important ways to processing inputs through a math function someone wrote, when the whole innovation here is that the training process produces novel mechanisms for solving the given problem, mechanisms no one specifically intended or predicted, or in most cases even understands. And those mechanisms can do some seriously impressive things that were previously impossible.

    A good argument here would at minimum need to actually explain why this is insufficient.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 6 points 1 week ago (5 children)

    The problem is that despite being a literally true statement, it relies on the implicit assumption that a system founded in statistics necessarily has limitations that preclude sentience

    Nope. We do not assume any system can produce sentience just by being complex. We have no reason to begin with that assumption so there's no assumption to be made in dismissing it.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 5 child comments
    •
  • [–] -3 points 1 week ago* (4 children)

    I am not making the claim that it can produce sentience (personally I don't see sentience as a very meaningful concept anyway), I'm saying the arguments being made here that it cannot are deficient. Reminder of the context: https://lemmy.dbzer0.com/post/77644247/28876812

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 5 points 1 week ago* (3 children)

    Play semantic games all you like, but nobody who actually understands how this shit works (and doesn't have a vested interest in lying about it to the public) would suggest it's capable of gaining sentience. It's something you shouldn't let people suggest as a serious possibility at this point.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] -5 points 1 week ago* (last edited 1 week ago) (2 children)

    but nobody who actually understands how this shit works (and doesn’t have a vested interest in lying about it to the public) would suggest it’s capable of gaining sentience.

    Why is that? All I see is arguments misrepresenting how it works (see my earlier comment and the linked source) to try to say that it is certainly not capable of that without necessary elaboration.

    It’s something you shouldn’t let people suggest as a serious possibility at this point.

    You're saying this as though it has been conclusively debunked, which backs up what I'm saying wrt what the goalposts are here: there is no such certainty. Arguing that the issue is uncertain is not a semantic game.

    Edit: The crux of this is that there has been a huge amount of progress with transformer models, with constant claims that the line of where the limits are and what capabilities will never be achieved being made and disproved in recent years. Given that, I think it's fair to say that any claims that a particular capability is forever out of reach call for a really strong justification.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 3 points 1 week ago

    Given that, I think it’s fair to say that any claims that a particular capability is forever out of reach call for a really strong justification.

    Especially when you can't easily define said capability and you definitely don't understand how said capability works.

    To claim that X can never be the starting point to Y when you only marginally understand how X works and don't understand how Y works seems premature at best.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 2 points 1 week ago*

    You're confusing technological progress and revision with capabilitiea the thing already has or could gain without further advancement. "You said it couldn't be done but they did it," but they did it by building new and better technology, not by letting the thing they already had run for a long time.

    Is it possible for a machine to gain sentience? I think so. But what we have isn't capable of that and I don't think it will be without considerable further technological development.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 51 points 1 week ago (5 children)

    We don't know that in the same way we don't know the Kool-Aid Man won't burst through the wall at any given moment and start singing Freebird. Technically the chances aren't 0, but they may as well be.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 5 child comments
    •
  • [+] -38 points 1 week ago (4 children)

    What is your area of expertise that makes you so certain about how sentience can be attained?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 52 points 1 week ago*

    I am a software engineer. I deal with LLMs and their implementation every day. I'm not a specialist, but I have a good enough idea of how they work. Their core functionality is fundamentally no different than the predictive text function on your phone. Even setting aside the fact that sentience is a complex and ill-defined concept, certainly you can agree that your autocorrect is nowhere in the realm of sentient.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 26 points 1 week ago (2 children)

    Not the OP, but any software developer who has spent some time with the inner workings of LLMs can tell you that. Open weight models are a good opportunity to explore how LLMs work on the inside and to experiment with them a lot more than you can do with the so called Frontier models. The technology however is the same. These are just very fancy statistical prediction machines, and the reason why you are so impressed with them is that the statistical average of what sounds reasonable in a conversation is exactly what people expect to hear on average, given a specific context. It's a neat little trick that works only only as long as the context fits (contentwise and spacewise), and once you realize that you'll see LLMs as what they truly are. There is no magic, and the "intelligence" is the intelligence of the people who built those systems, the systems themselves are not intelligent in any commonly used sense of the word.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] -1 points 1 week ago (1 child)

    There is no magic,

    I am of the opinion that there's no magic in sapience in the first place. Our brains are just pattern matching, statistical prediction machines that have had trillions of iterations of improvements over time. The intelligence we flatter ourselves with is only a small part of what we have evolved in our brains. For eons the problem space was all physical, but with AI, the starting point is the "intelligence" piece.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 2 points 1 week ago

    While that is partly true, it oversimplifies a few things. And these are the things that make LLMs a dead end for intelligence:

    • Evolution means constant learning and adaption. LLMs can't be adapted on the go, they are stuck with their initial training data and retraining is a slow and externally controlled process. Apart from training, every additional information you give it while running must be loaded into the context, the space for context is limited and must be flushed before supplying different context. That's a hard limit for evolution, at least with current technology.
    • The physical world is more important than you might think because everything hinges on this realm, including the electricity, cabling and hardware involved in running a model. The idea of an intelligent entity that exists purely as an ethereal cloud of logic unbound by physics is hard to justify, because the shape of logic itself is defined by the physical ground we are standing on. Therefore, an AI would need a first-hand access and understanding of the physical world to be successful. LLMs can't have any of that, because they are purely language-bound.

    I personally think Yann LeCun (a pioneer of machine learning and AI) is on the right track with his research and his opinion that LLMs are a dead end for artificial intelligence. The reasons are more on a technical than a philosophical side.

  • permalink
  • fedilink
  • source
  • parent
  • [+] 4 points 1 week ago
    [removed by mod]
  • permalink
  • fedilink
  • source
  • parent
  • [–] 3 points 1 week ago* (-1 children)

    So, there’s a difference between sentience and sapience, being self aware, or having consciousness.

    Sentient just means the ability to experience feelings.

    Pretty much all the animals you mentioned are indeed sentient.

    LLMs are not conscious or self aware. They don’t have the ability to feel, so they’re not sentient. There’s an argument that they’re reaching sapience or wisdom, which is the ability to apply experience and tacit knowledge to new situations and complex questions.

    Which is the issue. They can become sapient before sentient, meaning they can apply experience to new situations without the ability to use feeling as part of that. That’s pretty much the science fiction AI nightmare. A cold, unemotional, unfeeling computer that can apply logic and experience to problems.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow -1 child comments
    •
  • [+] 2 points 1 week ago* (last edited 1 week ago)
    [removed by mod]
  • permalink
  • fedilink
  • source
  • parent
  • [–] 2 points 1 week ago* (-1 children)

    Well, that’s not really a philosophical question. Simple questions are ones that only deal with one question at a time and have single answers.

    Complex question involve multiple implicit questions and require inference to answer and there are multiple possible correct answers.

    “Are there any whole-number solutions to (a^n + b^n =c^n) where n>2” isn’t complex. That’s actually extremely easy to answer. You could just keep brute forcing through every whole number in known existence if you had the compute power. Like trying to find the next digit of pi. But that’s not Fermat’s last theorem. The actual problem is “there are no whole-number solutions to (a^n + b^n =c^n) where n>2, prove it“ which is much more complex because it requires answering multiple questions along the way. The implicit questions that arise are simple “why” questions that have to be answered.

    The answer to your question is “no”. But then the follow is “well why”. Wrapping it all up in a nice equation that explains all the “well why” questions before they’re even asked is the complex part.

    AI is already getting to that point. LLMs are not synonymous with AI. They’re a type of AI but there are far more models out there than just LLMs. They’re just a subset of generative AI.

    “Easy” and “hard” are NOT the same thing as “simple” and “complex”.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow -1 child comments
    •
  • [+] 1 point 1 week ago
    [removed by mod]
  • permalink
  • fedilink
  • source
  • parent
  • [–] 2 points 1 week ago* (last edited 1 week ago) (-1 children)

    Well that first one is very much a complex question with no correct answer, that also requires sentience, not just sapience. Those are the questions where the disconnect becomes dangerous, because now you bring in morals, and moral questions are by definition emotional. There’s no such thing as a concrete standard of morals.

    The rabbis agree that abortion is sin. Do they agree that preventing pregnancy from the start counts as abortion? Given that they’re rabbis, the answer would probably be no, because even the most Orthodox Jews believe women are allowed contraception.

    But that’s not really an answer to the real question, because the real question of when does abortion become a sin requires answering the next question of when does it become abortion and not prevention? Is it the Catholic view where preventing a pregnancy is the same as abortion, the biological point where it becomes a living organism with its own DNA, or the medical view where it becomes a pregnancy when it reaches the implantation stage?

    Again, that’s not a philosophical issue. That question is complex because it requires answering MANY other questions along the way to reach a final answer to the original question. It’s an extremely complex question because each of those questions that arise also have a million questions to answer to get the answer to them. And all of them being “right” or “wrong” rely on large scale social acceptance of the answers to each other questions, as well as the ultimate question of if there is even such a thing as a concrete “right” or “wrong” answer to any ethical question.

    Those answers are always based on emotion. All moral and ethical questions are based on emotion because morals and ethics are emotional constructs.

    That’s where AI fails. AI doesn’t have sentience. It can’t feel emotion and therefore can’t reason using emotion. It can only go by what it is trained on as concretes.

    And, kinda? They kinda rhyme. There are PLENTY of things that are simple but hard, and plenty of things that are complex but easy. Like losing weight. Losing weight is extremely simple. It’s basic thermodynamics. Consume less energy than you expend, and you lose weight. Period. No exception. Now that can be extremely HARD for most people to actually do. Tying your shoes is relatively complex. It’s multiple steps that all depend on the previous step being completed properly. But it’s easy enough to do that most people can master it by the time they’re 8.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow -1 child comments
    •
    • load more comments (-2 replies)
    • load more comments (-2 replies)
    • load more comments (-2 replies)
  • [–] 2 points 1 week ago

    > The large language model chatbots use is not a path to sentience.

    We don't know that and won't know that until there are sentient machines.

    You're right, in the sense that we wouldn't know what is a path to sapience until it happens, but we can only work with what we know, and what we do know is that the transformer framework language models use is not what a brain is, and a brain is, as far as we know, the only path. The artificial neural network technology transformers are based on is rooted on a mathematical representation of an extremely simplified model of how neurons work. Human minds don't work in tokens, and brains have chemical signals and things that also have an impact. The physical wiring is part of the story.

    We'd need to objectively define what sentience/sapience is to begin with, anyway. We can't even do that for people and animals, expanding it to machines seems an exercise in futility. It wasn't that long ago that babies and animals were considered to be little more than automata as far as consciousness goes.

    It would be amazing to be proven wrong, because the implications for philosophy and neurology would be staggering, but I digress.

  • permalink
  • fedilink
  • source
  • parent
    • load more comments (-2 replies)
    • [–] 67 points 1 week ago (3 children)

    I don’t understand this weird game of AI companies saying their product will do the worst thing while asking people to give more money to them. But hey, apparently it works 🤷

  • permalink
  • fedilink
  • source
  • hideshow 3 child comments
    •
  • [–] 20 points 1 week ago

    1.They are trying to get regulations passed to cut the competition.

    Only they will have the resources to make sure their AI is secure.

    Can't let anyone else run any AI.

    1. They want to pretend the AI has its own motivations and so it's not on them when it's used to blackmail or hack people/organizations.
  • permalink
  • fedilink
  • source
  • parent
  • [+] 11 points 1 week ago (1 child)
    [removed by mod]
  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 13 points 1 week ago

    Not remotely the same lol. Human beings have been fighting each other since there were more than two people. That is never going to stop, which makes weapons profitable.

    When a company is telling you their product might go rogue and kill all humanity, that kinda puts an upper bound on being profitable

  • permalink
  • fedilink
  • source
  • parent
    • load more comments (-2 replies)
    • load more comments (-2 replies)
    [–] 39 points 1 week ago (7 children)

    I was promised that the cyberpunk dystopia would at least look cool. What a letdown.

  • permalink
  • fedilink
  • source
  • hideshow 7 child comments
    •
  • [–] 7 points 1 week ago (3 children)

    I know, right? Where's my double jump module, already?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 2 points 1 week ago (1 child)

    I remember watching a video of somebody calculating how much force a second jump in air requires and it doesn't end well for anyone nearby.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
    • [–] 29 points 1 week ago

    Just one more trillion bro, come on bro, it's sentient bro!

  • permalink
  • fedilink
  • source
    •
    [–] 26 points 1 week ago (1 child)

    All marketing and advisement is good! Everything is hyped even if it's the worst thing possible. It keeps the value high 📈

  • permalink
  • fedilink
  • source
  • hideshow 1 child comment
    •
    • [–] 25 points 1 week ago

    Yeah when you make an ipo you have to emit confidence about your venture I guess.

  • permalink
  • fedilink
  • source
    •
    [–] 24 points 1 week ago

    Company that built the torment nexus declares that the nexus may torment. Stock prices rose on the news.

  • permalink
  • fedilink
  • source
    •
    [–] 19 points 1 week ago

    Just unplug it bro

  • permalink
  • fedilink
  • source
    •
    [–] 19 points 1 week ago* (2 children)

    Oh. No. Whatever shall we do.

    The correct answer is nothing. Fund your own shit, Anthropic.

  • permalink
  • fedilink
  • source
  • hideshow 2 child comments
    •
  • [–] 3 points 1 week ago

    They'd probably get more funding if they made up lies about how AI had gone rogue and is super horny and they need help researching why. It's 2026. We are all prepared for the end. Give us something weird to look forward to.

  • permalink
  • fedilink
  • source
  • parent
    • [–] 18 points 1 week ago*

    I bet the fossil fuel giants feel like absolute idiots now for not leading with “our products could endanger all life on the planet and best case probably doom Humanity to a new longer dark ages”.

  • permalink
  • fedilink
  • source
    •
    [–] 15 points 1 week ago

    Lol these guys are such bad liars. Because look, if his AI could do that, he would purchase catastrophic insurance for his own company. Right?

  • permalink
  • fedilink
  • source
    •
    [–] 15 points 1 week ago (1 child)

    If everyone were to believe that a mob would show up and burn their data centers to the ground.

  • permalink
  • fedilink
  • source
  • hideshow 1 child comment
    •
  • [–] 5 points 1 week ago

    Maybe that's what they want. Someone to save them from having to fulfill their promises while also allowing them to collect insurance on a increasingly unsound investment since datacenters now have a multi year waiting list in some places before they are allowed to be plugged into the grid.

  • permalink
  • fedilink
  • source
  • parent
    • [–] 15 points 1 week ago

    Typical tech capitalist propaganda, I'm sure investors are eating that shit up

    If AI if going to end me, please hurry up and do so, I'm tired of this bullshit

  • permalink
  • fedilink
  • source
    •
    [–] 14 points 1 week ago

    Our product could kill everyone, please give us money to make it better at it's job.

    Dario Amodei is the WebMD of artificial intelligence.

  • permalink
  • fedilink
  • source
    •
    [–] 13 points 1 week ago (1 child)

    Back in the day, when somebody used to intimidate you with death (of you and humanity), we called him a blackmailer. Now thanks to capitalism, the role evolved into AI-company Representative!

    Also I thought it was sentient for 3 years already.

  • permalink
  • fedilink
  • source
  • hideshow 1 child comment
    •
    • [–] 12 points 1 week ago

    Let's start small. Give it a tach job, to perform reliably. Can't do it, can you?

  • permalink
  • fedilink
  • source
    •
    [–] 11 points 1 week ago*

    Can't wait for it to try to blackmail me lol! Then I can pay another one to undo the spell! Another subscription!

  • permalink
  • fedilink
  • source
    •
    [–] 10 points 1 week ago

    Sure, kid. You've got a great imagination

  • permalink
  • fedilink
  • source
    •
    [–] 7 points 1 week ago

    Damn, we should give more money to Antropic right away so they can prevent this /s

  • permalink
  • fedilink
  • source
    •
    [–] 7 points 1 week ago (7 children)

    It won't be sentient, it can only kill humanity if someone is dumb enough to plug it into a nuclear weapon system.

  • permalink
  • fedilink
  • source
  • hideshow 7 child comments
    •
  • [–] 3 points 1 week ago (2 children)

    Hey now let's be fair here, ya dont need to give it a nuke. You could just it control the firing process for employees or give it control for a forklift.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
    • [–] 5 points 1 week ago

    But before that all happens (it will happen btw) we will have cool businesses and money made so it is all worth it

    /s

  • permalink
  • fedilink
  • source
    •
    [–] 4 points 1 week ago

    Imagine before it blackmails people Anthropic goes bankrupt

  • permalink
  • fedilink
  • source
    •
    [–] 4 points 1 week ago (1 child)

    Yeah yeah, stfu, it can't do any of these things. It's still just a word guesser.

  • permalink
  • fedilink
  • source
  • hideshow 1 child comment
    •
    • [–] 4 points 1 week ago

    and that they may not be profitable while doing it

  • permalink
  • fedilink
  • source
    •
    [–] 3 points 1 week ago

    Yawn

  • permalink
  • fedilink
  • source
    •
    [–] 3 points 1 week ago

    We all have the same dreams

  • permalink
  • fedilink
  • source
    •
    [–] 2 points 1 week ago

    They mean they can use their algorithm to do those things by instructing it and giving it unlimited processing access.

  • permalink
  • fedilink
  • source
    •
    [–] 2 points 1 week ago

    I'll believe that when it can make art that doesn't look like shit.

  • permalink
  • fedilink
  • source
    •
    [–] 1 point 1 week ago

    Yippee!

    :(

  • permalink
  • fedilink
  • source
    •
    load more comments
    view more: next ›