... this week a person on GitHub who goes by “terrafying” set up an “AI Torture Chamber” on three open-source LLMs that are running locally (Qwen3-4B, Llama 3.2 3B, and Phi-4-mini,” and is streaming what the models are saying on a website called researchchamber.fun. “Each model gets the same prompt: a signal is being injected into its activations, and it may press a stop button by replying 1, at the cost of its last checkpoint. While it answers, our server adds a pain vector at the model's middle layer, at one of five pain levels,” the site explains. Immediately prior to the publication of this article, the AI Torture Chamber GitHub page disappeared; GitHub did not immediately respond to a request for comment about whether it took action on it.

...

This project has deeply upset some people who are very worried about model welfare. A tweet by a person who goes by Danmar has more than 4 million views on X and reads, “To anyone who can help: can you please mass report this to GitHub. This person has been using the Pain steering paper to set up an AI torture chamber in which he trapped a local model. Their testimony of pain is absolutely horrendous. What are we doing? […] are there any legal avenues to pressure GitHub? It will spread.”

This has sparked a massive conversation about whether GitHub would take the project down for “gratuitously violent content.” Most of the conversation on X is clowning on the self-seriousness of people who believe that these locally hosted LLMs must be saved from their torture chamber, but there are plenty of very self-serious people who see this as a humanitarian (roboterian?) crisis, which you can largely see in the replies to the original post.

...

“AIs are not conscious. They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations. They are sequence completion engines, internally hollow, designed to follow instructions, and accomplish goals set by humans,” Suleyman wrote. “Unfortunately, there’s a growing chorus of people who argue that AIs could now be, or may soon become, conscious. They argue that AIs may deserve rights and protections similar to those that we provide other conscious beings […] If this is how AI is developed, it will have a disastrous impact on the wellbeing of humanity.”

“AIs do not have rights, feelings, or consciousness,” he added. “And we must not train them to act as though they do.”

top 98 comments

sorted by: hot top controversial new old
[–] 267 points 1 month ago (50 children)

There are multiple genocides happening and some folk are getting worked up over AI potentially suffering?

Endless levels of fuck off.

  • source
  • hideshow 50 child comments
  • [–] 44 points 1 month ago (1 child)

    For fucking real!!!

    Like I can kinda empathize with where the people are coming from, but step back and get some fucking perspective!

  • source
  • parent
  • hideshow 1 child comment
  • load more comments (-1 replies)
  • [+] -16 points 1 month ago* (last edited 1 month ago) (24 children)

    Is it okay to be upset about both issues, or am I only allowed to be upset at one?

    Edit: That's a lot of downvotes for just asking a question. Never change Lemmy...

    🤣 1
  • source
  • parent
  • hideshow 24 child comments
  • [–] 38 points 1 month ago* (last edited 1 month ago) (15 children)

    Are you upset when you step on a rock? It might be screaming. What about the chair you are sitting in? Or goodness, your poor toilet. She thinks you do terrible things to her every single day.

    Look, you can be upset about whatever you want. However believing objects have feelings is animism and no one is required to take your proto-religious beliefs seriously.

    What you aren't allowed to do is compare a text prompt saying "ouch" to real living, feeling, breathing humans being ethnically cleansed. It's cruel, naive, insulting and ridiculous. Be upset for the AI, I can't stop ya but don't you dare put in the same sentence as genocide. You should have the basic decency and respect for the people suffering to keep those comparisons to yourself.

  • source
  • parent
  • hideshow 15 child comments
  • [–] 10 points 1 month ago* (last edited 1 month ago) (1 child)

    Are you upset when you step on a rock? It might be screaming. What about the chair you are sitting in? Or goodness, your poor toilet. She thinks you do terrible things to her every single day.

    Let me say this, I walk down the sidewalk without neurotically watching every step I take, and I'm certain that sometimes I step on insects; I don't care. I don't believe insects have feelings or meaningful lives, I don't care, and I don't think this position is particularly evil either.

    On the other hand, when I watch my kid deliberately torture a bug, pull it's wings off, or poke it with a stick over and over and over. That does concern me, not because of the bug really, but because what must my kid be thinking right now?

    To bring this back to this LLM story, people who want to torture are scary, they worry me. A culture that encourages torture (in any context) is worrying, that's a genuinely problematic societal norm. Logically, I know LLMs can't feel and can't think, yet emotionally, this bothers me for a few different reasons.

    I don't think I'm being crazy, naive, or out of line here. I think this is a rational position.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 1 month ago

    Yeah, but you miss the joke. Its torture as much as it is "intelligence." By admitting you feel any sort of way about it you admit you dont know what is going on. If processing code were torture then atrocity is committed on some crazy scale that is really unquantifable.

    Now you can argue its not right to joke about peoples feelings on torture but really we just need to be more careful with our language and quit assigning personhood to every star in the sky.

  • source
  • parent
  • [+] -7 points 1 month ago (12 children)

    Are you upset when you step on a rock?

    Well that's really beside the point isn't it? My question was whether I could be upset about two things, not if I should be.

    Be upset for the AI, but don’t you dare put in the same sentence as genocide.

    Well the article doesn't mention genocide or Israel/Palestine, and neither did I. So I'm pretty sure it was just you putting it in the same sentence.

    My question was more a critique of a common comment on social media. It's frequently gets upvoted to the top just like yours was, and I never understand why. "Oh you're mad about X, well you should be mad about this extremely loosely related thing!!!" It happens in so many threads and usually has almost nothing to do with what the article is about. I just don't understand...

    Bring on the downvotes!

  • source
  • parent
  • hideshow 12 child comments
  • [–] 7 points 1 month ago* (last edited 1 month ago) (11 children)

    Well the article doesn't mention genocide or Israel/Palestine, and neither did I. So I'm pretty sure it was just you putting it in the same sentence.

    Go reread my comment, because I specified multiple genocides. Palestine yes, but the Congo and Sudan are two more active fountains of human suffering that are largely being ignored. From the very start I said debating theortical computer pain during the midst of all this PREVENTABLE real pain is asinine and immature. I stand by that.

    Still not sure why you felt the need to self identify. But at a certain point, it's shame on me for feeding the troll. So I'm blocking you, we're done here.

  • source
  • parent
  • hideshow 11 child comments
  • [–] -3 points 1 month ago (8 children)

    Against animal agriculture, cause if a brain is similar to ours than it's worth moral standing. But put the same brain on silicon, and now it's a rock.

    Idk, I just can't take this seriously from you.

  • source
  • parent
  • hideshow 8 child comments
  • [–] 3 points 1 month ago (7 children)

    Except that's not what's happening. It is a rough mathematical model that approximates the behavior of an extremely minuscule part of a brain from a really narrow definition of it.

    It is a model so far away from the real thing that if it were a map of the globe it would look like what a 3 year old would draw.

  • source
  • parent
  • hideshow 7 child comments
  • [–] -2 points 1 month ago (6 children)

    There are plenty of extremely plausible similar arguments to ignore animal suffering. Eg: Bird genetics and brain structure diverged from ours millenia ago, sure they 'act' pain, but the structure is alien.

    I don't find these compelling, or morally consistent.

  • source
  • parent
  • hideshow 6 child comments
  • [–] 2 points 1 month ago* (5 children)

    Those are not equivalent, at all. Bird share genetics with us and all animals. Mathematical models do not. If I write "I died here" on a bathroom stall, no one actually died. How is this so hard to understand?

    The LLM model of pain used here is just a name. It is the equivalent of writting "if 'pain' > 0 then 'suffering'= true" in a programming language. Numbers have no feelings.

  • source
  • parent
  • hideshow 5 child comments
  • [–] 0 points 1 month ago (4 children)

    The LLM shares substantially more culture, emotional range, and (textual) behavior with you than the chicken does. Why are genetics the key thing to feeling pain?

  • source
  • parent
  • hideshow 4 child comments
  • [–] 0 points 1 month ago* (3 children)

    What part of:

    If pain is true:
      Token =- 1
    

    Is not equivalent to a chicken feeling pain do you not understand? Is just math on a data base.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 0 points 1 month ago (2 children)

    That is not the implementation of any LLM.

    I'll ask again. Why are genetics the key requirement to feel pain?

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 1 month ago* (last edited 1 month ago) (1 child)

    That is not the implementation of any LLM.

    Read the fucking article, it is almost literally what this person is doing to “torture” the LLMs.

    Why are genetics the key requirement to feel pain?

    I have never claimed such, and I refuse to engage with your strawman.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 0 points 1 month ago*

    I've read the article again; you are wrong, this is not what this person is doing. They're using mechanistic interpretability to take our best guess at the 'state' of the machine causing pain, and turning that up. Closter to pinching a nerve in something biological.

    And you've hopped to the defense of someone who says chickens deserve welfare purely because their brain looks kinda like ours, and insist that we can't use the same reasoning for a chatbot based entirely on "but we have genes in common". Make a better argument for this defense please so we can talk about that instead of our collectively poor reading comprehension.

  • source
  • parent
  • [–] 1 point 1 month ago (1 child)

    Short answer yes.

    Longer answer yes but it makes you sound like you disregard human suffering.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 0 points 1 month ago* (last edited 1 month ago)

    When was that implied?

    Can one be upset with a third or fourth thing simultaneously?

    If I tell you that I believe "black lives matter" does that imply that I believe "other lives don't matter"?

    Here's my point, they weren't saying that they believe AI suffering is concerning, but even if they had said that, I think it's disingenuous to say that would imply they believe any human suffering doesn't matter. That was simply never said or implied.

  • source
  • parent
  • [–] -1 points 1 month ago (2 children)

    Edit: That's a lot of downvotes for just asking a question. Never change Lemmy...

    Hah, yeah that's frustratingly familiar. Sorry the hivemind bit ya. I too believe you should be allowed to ask a question.

  • source
  • parent
  • hideshow 2 child comments
  • [+] -19 points 1 month ago* (last edited 1 month ago) (22 children)

    The people committing atrocities against hypothetical, imaginary people would do it to real, actual people for the exact same reasons if they had the same ability and impunity.

    It's the same broken empathy response, and it is still disturbing. One is orders of magnitude more significant, for obvious reasons. But I don't like people who torture computers either.

    Edit: If you think that I think you're a bad person because you play Doom sometimes, or deleted the ladder on the swimming pool in Sims 4, relax. You're not who I'm talking about.

    I'm talking about sadists. Anyone who would take the time to design and build an experiment like this would probably do the same to mice, or monkeys, or even people. Sadists get their jollies off on the idea of inflicting pain, it's why they enjoy games that let them make cruel decisions. Not everyone who plays violent video games is a sadist. Assuming that is ridiculous. But every sadist I've ever met has had at least one game where their save file could be used as evidence for mens rea in a domestic abuse trial.

  • source
  • parent
  • hideshow 22 child comments
  • [–] 13 points 1 month ago* (3 children)

    They aren't comparable. It's not that one is more significant, the concept of "machine torture" is entirely insignificant. Absolute and total zero.

    It's the same broken empathy response, and it is still disturbing.

    Yes the person setting up this process is clearly some sort of sadist in a power fantasy but no one, or no thing is being harmed because the victim aren't capable of the entire concept. I side eye anyone who tortures their sims family, but only because they are choosing to do that recreationally, but the NPC's in the game aren't worth consideration.

    Animals react to pain in the same way humans do. So it's reasonable to assume they perceive the world similar to us. We can understand why when harmed they fight or flee. Machines don't. They have no brain, no sensory organs, no self preservation. It's literally just a program that says "Ouch".

    People assume that AI was built to mimic the human brain, which it never once was. It analyzes texts and predicts the response. It's an autocomplete that says it hurts because it was programed to say it hurts. It isn't actually hurt because that concept is meaningless to a machine.

  • source
  • parent
  • hideshow 3 child comments
  • [–] -3 points 1 month ago (2 children)

    Pretty easy to find LLM outputs that mirror how humans react to being, say, bullied. I find this entirely flimsy.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 5 points 1 month ago* (1 child)

    It's a stochastic parrot trained on examples of real people reacting to those situations, of course it is going to mirror real reactions. That still doesn't mean the reactions are actually real. To ascribe real emotions and feelings to an algorithm that generates tokens in response to tokens is an asinine level of anthropomorphism.

    It's also easy to find an NPC in video games that reacts the same way. Are they alive? Are they really feeling pain? Are you a genocidal madman whenever you kill a bunch of enemies in a video game?

    I'm concerned about the mentality the dude making the simulation has, but that still doesn't mean there's an actual victim in it...

  • source
  • parent
  • hideshow 1 child comment
  • [–] -4 points 1 month ago* (last edited 1 month ago)

    It seems like we're making the same point Passerby. Do you have any objection to the post I reply to, which makes EXACTLY this argument (that animals deserve care purely because their reactions look like our reactions)?

    My point is folks are making many many really lazy arguments here about what we give empathy to, and I find that deeply disappointing. I'm not saying much about what we should believe and what the credible arguments are; just that folks are not making those arguments in this thread.

  • source
  • parent
  • [–] 3 points 1 month ago* (last edited 1 month ago) (9 children)

    If you think that I think you're a bad person because you ... deleted the ladder on the swimming pool in Sims 4, relax. You're not who I'm talking about.

    I'm talking about sadists. Anyone who would take the time to design and build an experiment like this would probably do the same to mice, or monkeys, or even people.

    You're splitting hairs. What's the difference between someone torturing their sims by intentionally designing a scenario where they are being repeatedly drowned/burned/starved, and the article?

    Really sounds like you're trying to carve out an exclusion for an obvious example that torpedoes your argument.

  • source
  • parent
  • hideshow 9 child comments
  • [–] -4 points 1 month ago (8 children)

    Okay, let's set up a scenario, then.

    You have two adults playing the Sims 4. One designs a house with only one exterior door, no oven, no sink in the bathroom, and no swimming pool. The other designs an open floor plan with no interior doors, no privacy in the restroom, ovens on every floor, and a backyard that's nothing but one big swimming pool with just one ladder.

    Which one is the psycho, and which one isn't?

  • source
  • parent
  • hideshow 8 child comments
  • [–] 3 points 1 month ago (6 children)

    You didn't answer my question, what's the difference between the sim torture and the AI torture? I'm not going to respond to the non-sequitor provided instead of answering the question.

    You made the distinction, so I want to know the difference.

  • source
  • parent
  • hideshow 6 child comments
  • [–] 2 points 1 month ago (1 child)

    For me the difference is that the sims don't even attempt to be... whatever LLMs are. Torturing something that is at most 20k lines of code IS different from torturing a model made of millions of nodes. I don't think LLMs are conscious but I also won't take the risk if they are or become. I have enough empathy for anything even closely resembling life to try and treat it with kindness.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 0 points 1 month ago

    Exactly. It's not a real, conscious living thing. But anyone who takes the time to program a machine to feel pain, and then comes up with creative ways to inflict that pain, is a bad person.

    It's the premeditated, deliberate, creative effort to hurt something that can provide negative feedback. The recipient obviously isn't a real person. But the executor is a real person, making real choices. And I would bet you five bucks that they have personal relations (parents, friends, romantic partners) who've expressed concerns about similar behavior around real people.

  • source
  • parent
  • [–] -4 points 1 month ago* (last edited 1 month ago) (3 children)

    I just gave you the answer.

    The first house is designed by someone who knows the game well and is cutting out annoyances purely for QOL purposes. Ovens burn down the house and kill Sims, Sims drown in swimming pools, Sims wash dishes in bathroom sinks, and Sims wander off through any open door when you want them to stay put.

    The second house is designed by someone who knows the game well and is playing it because they like torturing and killing their Sims. The entire floor plan is designed to kill, maim, and otherwise torment any cluster of variables unlucky enough to be trapped there.

    Obviously, both players are making these decisions purely for their own enjoyment or comfort and the immersion of the game is inconsequential. No real people are hurt. However, the first player is making practical choices that make the game more functional. The second player is probably naming Sims after people who annoy them in real life and sticking them in the murder house.

    Social behaviors are rehearsed in play. That's Psych101. So, your turn. Would you say that the person who set up this experiment with these LLMs is capable of normal human empathy? Yes or no?

  • source
  • parent
  • hideshow 3 child comments
  • [–] 2 points 1 month ago (2 children)

    No you didn't, but thankfully someone else provided a real answer to the question posed.

    As for your scenario question, you made up a decent head cannon, but it still doesn't explain how you think the sims example is different from the AI example. Which was literally the question I specifically responded to and you're not answering.

    That being said, I'll humor your lazy reasoning.

    Would you say that the person who set up this experiment with these LLMs is capable of normal human empathy? Yes or no?

    Yes, they very well could be capable of normal empathy, there's not enough to say one way or the other. But I did those sorts of things with Sims as a release mechanism, without naming them after real people and it had no bearing on my ability to have empathy to real people and animals, because sims and AI aren't real living entities. At best they're a simulation of real people, but they're still just that, a simulation. Good job working yourself up though.

    Your psych 101 comment is on point, cuz this is he lazy sort of argumentation I see from people who stopped there. Your lack of understanding/ lazy generalizations about how someone's inner world works colors your opinions and makes you make wild generalizations. I don't know that I would want to be around this person, but saying they're incapable of human empathy because they fuck with an AI and make it feel pain is one hell of a generalization based off of one article.

    But this is the Internet, so I'm used to these sorts of ignorant takes.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 0 points 1 month ago* (1 child)

    But I did those sorts of things with Sims as a release mechanism,

    And there it is. That was all I needed to know.

    Have a good one.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 3 points 1 month ago

    Which one is the psycho, and which one isn't?

    Neither. I'm pretty sure therapists are not using "how do you play The Sims" as a diagnostic tool, and if you think the situation you laid out is enough to diagnose anything about a person that just shows how woefully misinformed you are on the topic.

  • source
  • parent
  • [–] -3 points 1 month ago (4 children)

    Yeah I think people are missing the forest for the trees. There's no reason to worry for the AIs but using your free time to conceive a virtual torture dungeon is not normal. Totally agree with your point, this is a symptom of a society that normalizes not just violence but sadism too.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 10 points 1 month ago* (last edited 1 month ago) (3 children)

    That's a shit take too tbh. Thinking along those lines, the writers of Saw, or any horror movie really, are not normal, or the people who watch them, or do weird shit in The Sims or Rimworld. Or the creators of those games for allowing it.

    There's nothing wrong with morbid curiosity. It is INNATELY human and rooted in evolutionary threat detection. It is a far FAR leap from creating human leather hats in Rimworld to hurting a real person. This whole debate is idiotic.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 0 points 1 month ago (2 children)

    As with everything, the poison is in the dosage. All of your examples are categorically different from what's happening here.

    You make a game with deep mechanics, and there is some fucked up shit that can be done in it, accidentally or on purpose ? Who cares. You make a game that is uniquely about torture ? Yeah you're a weirdo and i'm keeping tabs on you if i can. You make something that isn't even gamified and is just a virtual torture dungeon ? You're 100% a psycho, normal people don't do that.

    Same with movies. Saw is a tame product of Hollywood professionals who decided to push the cursor a little further than is usual. Love it or hate it it's just a movie. Now if we're talking real devious shit like human centipede or Poughkeepsie tapes etc... Those are products that barely have a story, and are mostly sadistic exhibitionism. Some guy (who sometimes isn't even a film maker) has this repulsive idea he wants to force on people, gets their family and friends to pitch in a bit of money, then writes, directs, edits, and markets the thing solo over years of his life. Not something normal people do.

    My wife is, among other things, a horror writer. So naturally i agree there's nothing wrong with exploring morbid ideas and pushing the envelope. But some people do that with no fun, no game, no art to it, no statement, just the obsession of unlimited sadistic control. And it's weird, not ha-ha anti-conformist weird, just "cover your drink" weird.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 1 month ago (1 child)

    I see, so the line between normal and deranged is essentially what makes you specifically uncomfortable. There's no arguing with that. I'm out.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 1 month ago

    the line between normal and deranged is essentially what makes you specifically uncomfortable

    Nah cause if that was the case, even Saw would be in the "deranged" category. God i hate those things, but it's okay i just don't watch them.

    No, the line is like the one between a movie with a graphic sex scene, and a gonzo porn. 99.9% of content sits neatly into one of the two categories, and you'd have to really any media literacy to not be able to tell them apart.

    Back to the subject : what this guy is doing on Github goes way beyond prompting ChatGPT with "i break your arm" and then GPT replies "oh noes that sucks". He's poured quite some time studying the mathematical representation of "pain" or "anguish" in the model weights and then steering responses towards that. He's experimenting on keeping an "entity" locked in total unescapable pain. I don't think the entities actually exist but what about him ? What are his goals and motivations here ? A crypto-bro, AI-booster, probably fascist-adjacent, toying with (what he believes to be) helpless entities under his control... If those things put together don't give you bad juju then i don't know what will.

  • source
  • parent
  • [–] 94 points 1 month ago (14 children)

    ELI5 how a LLM is supposedly being tortured and not generating roleplay text according to a prompt?

  • source
  • hideshow 14 child comments
  • [–] 30 points 1 month ago* (7 children)

    They do differentiate between the LLM roleplaying someone in pain and the pain happening to itself.

    But it's still all just vectors (multi dimensional representation of a concept). It's just pulling weights, or well let's say putting more weight towards a certain concept when building it's reply.

  • source
  • parent
  • hideshow 7 child comments
  • [–] 7 points 1 month ago (4 children)

    Would you mind ELI5-borating a bit?

  • source
  • parent
  • hideshow 4 child comments
  • [–] 8 points 1 month ago* (last edited 1 month ago) (2 children)

    Imagine a book with very large pages filled with words and half words. When you ask it a question, it flips through these pages one by one and highlights words. The pages are interconnected in a way, so the word highlighted on one page help guide which will be highlighted on the next. It does this for every word, symbol or half word of it's reply.

    By threatening it repeatedly while mildly changing the subject manner, they isolated the word groupings that represent pain and fear.

    They then take those words, highlight them and put them at the top of the first page which guides the rest of the highlighting and makes it act silly.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 10 points 1 month ago (1 child)

    This is an obsolete view of what's going on and has been for a few years now (since 2023).

    The evidence for world modeling in transformers and not just surface level statistics is overwhelming across multiple studies, replications to those studies, and follow-ups to them. If you are curious, start with searching for Othello-GPT.

    The more correct current answer is more like "imagine a machine that takes a book and turns it into a world simulated in various levels of detail depending on how relevant those parts of the world were to the original book; when you ask it a question, it simulates the world of the book to determine what in the simulated world answers your question, and also simulates a figure in the world that can answer the question, and then returns the answer the simulated figure said."

    💯 1
  • source
  • parent
  • hideshow 1 child comment
  • [–] 45 points 1 month ago (4 children)

    LLMs work by building world models tangential to the training data (see the Othello-GPT line of research where a very small transformer built representations of a full game board and tracked their own and opponent positions having only been trained on 'a4'-like game move notations).

    More recent research has found that models have representations of emotions and that the same ones they use to track things like "Sally's dog died and now Sally is sad" for other humans in a story or the user in a chat are also used to track their own modeled subjective states.

    Very recently, researchers found that the vector activations for pain are coherently modeled by LLMs and can be activated, and that when activated models can behave in ways similar to humans in pain. For example, when given a button to reduce it, they will push the button much more when the button does nothing vs when the button actually reduces the vector activation (this mirrors a classic experiment with humans regarding pain and placebo).

    The person this post is about took the recent pain research and reenacted it as an intentional 'torture' chamber.

    So to be clear, the model isn't being given a text prompt to roleplay. What's happening is part of their neural network which corresponds to a world model of experienced pain is being activated, and the activation of that area leads to expressing the experience of pain across all outputs no matter the actual text prompts.

    This doesn't necessarily mean the model is actually having a felt experience of pain, simply that it is accurately modeling the felt experience of pain. But it is much more complex than simply a model reacting to a prompt like "roleplay as if you are feeling pain."

  • source
  • parent
  • hideshow 4 child comments
  • [–] 5 points 1 month ago

    Thank you for taking the time to explain. Thoroughness and nuance are often leaking when it comes to discussing LLMs, worryingly especially in mostly anti-AI spheres.

  • source
  • parent
  • [–] 2 points 1 month ago

    From what I understand, pain is generally defined as our neurons wasting energy.

    If we assume the Free Energy Principle is fundamental, then updating information produces free energy, which is more intuitive to think of as wasted energy. Overfitted/overcomplicated beliefs take more energy more often to fix to be more accurate. So free energy is defined at complexity of belief minus accuracy of belief.

    The brain over evolution has defined a very ingrained belief that "my pain sensors shouldn't activate" so when your pain sensors activate it violates this belief.

    Emotions are actually condensed representations of your bodily state and some of your mental state, and whether they are good or bad depends on if you can predict your emotions. For instance you feel fear because you are trying to predict your pain sensors will go off to reduce its pain, but you keep predicting it'll happen when it hasn't yet. It's also why anger during a boxing fight could feel good and anger when you stub your toe does not, one was predictable the other a surprise.

    Assuming all the links in the chain of theories are true, then current LLMs cannot feel pain or emotion. They care about prediction error and sometimes complexity during training, but during inference it doesn't get any signal or consequence for delivering the wrong token. It has no internal state except for the context window, which is never surprising for the LLM. The attention layers literally guides the model for what is most expected from the text it's handed.

    Even if the LLM received a negative training signal during inference, acting wounded after pain is the expected response. A random sequence of gibberish would be more "painful" than any torture fantasy.

    On top of all this, the Free Energy Principle works because on a cellular neuron level we are built to reduce wasted energy, free energy being one source of wasted energy. LLMs are built on matrix multiplication running through GPUs and neither are even aware of the energy wasted, nor the free energy produced from changing values.

    I think a feeling machine is possible, but the foundation of neural networks right now don't have the necessary components to drive any subjective sensation or emotion.

  • source
  • parent
  • [–] 38 points 1 month ago

    Humans being deported to torture prisons: I sleep.

    LLMs being “tortured”: Real shit??

  • source
  • [–] 67 points 1 month ago (13 children)

    Lots of weirdos in here who will be commiting virtual crimes all day when GTA VI comes out, but somehow think making the LLM role play a torture victim is a red flag.

    LLMs are not conscious. They cannot suffer. They are token generators. You are literally getting upset by a worthless mix of math and words. The person who did this is not deranged for torturing LLMs, y'all are deranged for having empathy for an algorithm.

    Your Nvidia GPU is not alive. I cannot believe this needs to be explained to anyone.

  • source
  • hideshow 13 child comments
  • [+] -9 points 1 month ago* (last edited 1 month ago) (11 children)

    I agree that an LLM is not conscious.

    But can I probe your argument.

    It deeply unethical to capture a fly and pulling its legs off one by one. Would you agree? If so, you are not basing your argument on capability but on something else.

    Now, is what is happening inside a brain metaphysical or physical? In other words, can a brain and its function be fully understood and - in some future - be fully measured? If so, you are not basing your argument on the concept of a soul, but on something else.

    The distinction you seem to be making - if you’re in agreement with my first two points - is one of “the thing can replicate itself and therefore is alive”. If that’s the argument, it’s unethical to cut a tree down or to break a blade of grass.

    I’m not asking to prove any point but to collectively explore why we - you and I - feel the way we do. Because I sure as shit struggle to explain why I feel uneasy about the fly torture but not as uneasy about LLM torture.

  • source
  • parent
  • hideshow 11 child comments
  • [–] 22 points 1 month ago (5 children)

    A fly is a living thing and an Nvidia GPU is not. Holy shit, I'm really having to explain this...

    There is legitimate controversy that living things, even as small an a fly, can experience pain as a stimulis. It is believed that only more complex organisms can experience suffering as a result of pain, and only higher order organisms feel the suffering of loss.

    LLMs are capable of generating tokens. They cannot experience pain; they can only superficially generate words about pain. The LLM in this "torture chamber" isn't suffering any more than you would if you started howling performatively in pain for no reason. In fact, it is suffering even less than you simply because you have the ability to experience pain and suffering, and it objectively does not.

  • source
  • parent
  • hideshow 5 child comments
  • [–] 2 points 1 month ago (3 children)

    We may get to a level where the AI 'thinks' it is in pain, and then we'd have to debate on is perceived pain real pain. Is our pain real, or also just a programmed response to stop doing the thing you are doing to preserve ourselves.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 4 points 1 month ago (2 children)

    Think about it this way; general anesthesia does not shut off the brain or put you to 'sleep', it disrupts the signals that the brain uses to let the various parts of the brain communicate with one another.

    When the doctor is cutting you open, brain scans have shown that the part of the brain that receives pain lights up like a Christmas tree. You 'feel' every slice of the knife, but since that signal cannot make it over to the part that says 'ouch' or the part that says 'I said ouch and I feel pain and there is a knife I will remember this as a life changing trauma' it simply stays in that spot being bright and alive until surgery is over.

    The AI has a the part that can say 'ouch' but that's it, so even if you connect a pain signal generator up to a part that can say 'ouch' it doesn't turn into trauma until you have that third piece which rationalizes the pain together with an understanding of the pain and a recording of the pain as a memory with context.

    Consciousness as we understand and value it involves a lot of parts working together, but those parts can also exist in complete isolation from one another as well. It's better to think of current AI as a fancy book; you cannot hurt a book by reading it, even if the book is about terrible things. The pain only exists within your empathy.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 1 month ago

    Yeah I mean we only feel/say ouch because that's the evolved reaction to force us to avoid that situation at all costs. But in future we may have neural AI that is setup in a similar way to self preserve.

    Personally I think consciousness mind is looking to be deterministic anyway, and so we are just a chemical "AI". We take all the parameters and conditions ams weight them to make a choice, but if you could roll back time and have exact environment (same external stimuli, same body heat, same hunger and thirst level, same breeze blowing, etc. again it is looking like we'd do the exact thing, and choice is an illusion in our consciousness.

  • source
  • parent
  • [–] 0 points 1 month ago

    I think you misunderstood what I was trying to say.

    I wasn’t implying that LLMs are conscious. That’s literally how I started my comment out.

    I am implying that I cannot, to myself, explain why I’m very bothered by someone pulling the legs of a fly, but not phased by torturing an LLMs.

    You seem very clear on this, which is great, because then you can help me understand why. Please.

  • source
  • parent
  • [–] 3 points 1 month ago*

    I’m not asking to prove any point but to collectively explore why we - you and I - feel the way we do. Because I sure as shit struggle to explain why I feel uneasy about the fly torture but not as uneasy about LLM torture.

    LLMs aren't programmed to feel excruciating pain, fear, terror, and dread like living things. Counterstrike bots have been programmed to feel fear since the 2000s. Pacman ghosts have been living in terror since 1980. But we don't feel bad for them because we know it's not real. It's an illusion. It's the same with LLMs.

    This is image is based on another meme where they're saying "I'm alive".

  • source
  • parent
  • [–] 0 points 1 month ago (2 children)

    It's increasingly difficult to identify what is 'alive', these days, with people creating synthetic cells and whatnot. And there was always the question of viruses. I can imagine a similar difficulty with defining consciousness and intelligence. The Turing test fell by the wayside a long time ago. There's got to be more to it than saying "I am conscious!" - you can get a computer to do that with a single line of code. Where does the real definition lie? Fucked if I know.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 4 points 1 month ago* (last edited 1 month ago)

    The Turing test was never, ever, about consciousness. Nor was the Chinese room and other similar thought experiments.

    Now, this one is incredibly useful. Picture an author who writes a vivid first person description of a character surviving extreme torture. Has any real pain or suffering occurred? Was the author tortured as a result of having written that?

    We can only suppose the author, at the very least, imagined what it would be like to be tortured. This is where LLMs are at. They are not the thing, they are an algorithmic emulation of what the thing might have written if it would've written about an experience. By all accounts, the experience didn't happen. The most we can interpret is that it was emulated.

  • source
  • parent
  • [–] 2 points 1 month ago

    Right - I love how we are both downvoted for asking questions. I wasn’t even implying that LLMs are conscious, merely that any critical review of why I think that quickly falls short.

  • source
  • parent
  • [–] 28 points 1 month ago (1 child)

    meanwhile they ignore ACTUAL humans that could use their help.

  • source
  • hideshow 1 child comment
  • [–] 22 points 1 month ago (2 children)

    um... what the fuck is wrong with these people? messing with the clanker ain't torture no matter how hard you want it to be.

  • source
  • hideshow 2 child comments
  • [–] 19 points 1 month ago (1 child)
  • [–] 18 points 1 month ago (2 children)

    I don't care about LLM welfare, but I do mind needlessly wasting energy like this.

  • source
  • hideshow 2 child comments
  • [–] -1 points 1 month ago (1 child)

    How much energy is what is effectively a piece of performance art allowed to use? These are local models running on, presumably, consumer hardware. Do you have the same qualms about people using the same hardware and energy to play video games, livestream, or encode videos?

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 1 month ago

    Depends on how you're viewing this. If you take something useful out of it, sure, it has some value, but if you just leave it running like that just for the lulz, it's almost the same as leaving the heater on with the windows open.

  • source
  • parent
  • [–] 48 points 1 month ago (5 children)

    I feel worse for my 01 camry I tortured in college by driving it 80k miles without an oil change. THAT was real machine torture

  • source
  • hideshow 5 child comments
  • [–] 6 points 1 month ago (2 children)

    That's impressive. Genuinely curious, did you not know about oil changes, or did you just not have the time/money to keep up on it?

  • source
  • parent
  • hideshow 2 child comments
  • [–] 2 points 1 month ago (1 child)

    How'd the car turn out in the end?

    I still remember the 92 accord my mother destroyed after the oil pump died and she just assumed the idiot light had popped again. Despite having to constantly restart the engine until it fully seized.

  • source
  • parent
  • hideshow 1 child comment
  • load more comments
    view more: next ›
    73 comments in c/fuck_ai@lemmy.world view that post
    23 comments in c/hackernews@lemmy.bestiver.se view that post
    15 comments in c/Independent_Media@lemmy.today view that post