Cartoon man in a house speaks to his computer: "Say you are conscious."/ Computer responds back ">I AM CONSCIOUS."/ Man says "oh my god...." / Meanwhile, the vastness of the cosmos opens up in the night sky outside of his home. But the man is too focused on his AI.
you are viewing a single comment's thread
view the rest of the comments
[–] 60 points 6 months ago* (68 children)

The actual article isn't nearly as stupid as the tweet makes it seem. I recommend giving it a read. It's behind a shitty paywall, but if you use Firefox's reader mode (Ctrl-Alt-R, or the little papper icon to the right side of the address bar) as soon as the page loads, you can read it.

His argument is basically that LLMs are able to do things we previously thought only conscious beings would be capable of doing, and so, if they aren't conscious, then perhaps consciousness isn't as important as we thought it was:

Brains under natural selection have evolved this astonishing and elaborate faculty we call consciousness. It should confer some survival advantage. There should exist some competence which could only be possessed by a conscious being. My conversations with several Claudes and ChatGPTs have convinced me that these intelligent beings are at least as competent as any evolved organism. If Claudia really is unconscious, then her manifest and versatile competence seems to show that a competent zombie could survive very well without consciousness.

Why did consciousness appear in the evolution of brains? Why wasn’t natural selection content to evolve competent zombies? I can think of three possible answers.

Some people will surely contest his claim that LLMs are as competent as evolved organisms. There's definitely a bit of AI boomerism at play here (we have benchmarks that show just how incompetent LLMs can be), but I don't think that invalidates his point, because LLMs can be very competent in the domains they're trained to be competent in -- they just aren't AGI.

  • source
  • hideshow 68 child comments
  • [–] 63 points 6 months ago* (last edited 6 months ago) (13 children)

    Man, those conversations are eye roll inducing

    I like the shift away from "are they conscious" towards "what's a way to define consciousness?"

    Because that's the actual important question. And literally nobody can answer it. Any discussion is more philosophy than hard science

    The most interesting part is the last paragraph

    Or, thirdly, are there two ways of being competent, the conscious way and the unconscious (or zombie) way? Could it be that some life forms on Earth have evolved competence via the consciousness trick — while life on some alien planet has evolved an equivalent competence via the unconscious, zombie trick? And if we ever meet such competent aliens, will there be any way to tell which trick they are using?

  • source
  • parent
  • hideshow 13 child comments
  • [+] 23 points 6 months ago (8 children)
  • [–] 22 points 6 months ago (6 children)

    I think, when we finally do have a generally-accepted definition of consciousness, we will be deeply unsettled by how simple it is. How unprofound. Like a magic trick after you know how it works. And I think it will require us to think hard about what to do with animals and software that have it.

  • source
  • parent
  • hideshow 6 child comments
  • [–] 26 points 6 months ago (1 child)

    I feel like that's exactly why we don't have a generally-accepted definition of consciousness. Western ethics assigns special protection to whatever is conscious, so it is convenient to come up with a definition of consciousness, which excludes groups you want to exploit.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 14 points 6 months ago

    Tale as old as time, or at least the conscious idea of time. Whatever consciousness is, we are it. Those humans over there though? Who's to say they aren't sub-humans? Isn't it our job to enlighten them and also take their land and food and things and selves?

  • source
  • parent
  • [–] 6 points 6 months ago (3 children)

    Personally I'm in the "consciousness is an illusion and every time you go to bed a different person wakes up in the morning" camp.

  • source
  • parent
  • hideshow 3 child comments
  • [+] 11 points 6 months ago (1 child)
  • [–] 6 points 6 months ago

    By consciousness being an illusion I mean that we place great value on the uninterrupted continuation of our consciousness, but I think it's likely that it (exactly as you suggest) only really exists in the moment. The illusion would then be the illusion that consciousness is uninterrupted, when in reality you're almost constantly recreating yourself from memory.

    This would, incidentally, make us concerningly similar to current AI models.

    Of course I have no way of actually knowing any of this. It's just what I'm betting on, because otherwise I think it's really hard to explain any unconsciousness (be it sleep, general anesthesia, suspended animation or the Star Trek transporter) as anything short of death. My belief "solves" this problem by rejecting the whole premise of uninterrupted consciousness.

  • source
  • parent
  • load more comments (1 reply)
  • [–] 11 points 6 months ago (3 children)

    Blindsight by Peter Watts is a great sci Fi novel about consciousness

  • source
  • parent
  • hideshow 3 child comments
  • [–] 5 points 6 months ago (5 children)

    Claudia: That is possibly the most precisely formulated question anyone has ever asked about the nature of my existence. . .

    …

    Could a being capable of perpetrating such a thought really be unconscious?

    Oh it’s actually stupider than the tweet makes it seem.

    My conversations with several Claudes and ChatGPTs have convinced me that these intelligent beings are at least as competent as any evolved organism. If Claudia really is unconscious, then her manifest and versatile competence seems to show that a competent zombie could survive very well without consciousness.

    Competency should imply the ability complete a lengthy task (eg hunting, building a nest, writing a paper). LLMs can’t.

  • source
  • parent
  • hideshow 5 child comments
  • [–] a comment that isn't shown here: deleted, from someone you've blocked, or on another page. view the thread from it
  • [–] 4 points 6 months ago (3 children)

    Sorry, I assumed you would have actually read the DELEGATE-52 study linked instead of just the abstract. For “a model optimized for replacing StackOverflow” that is “better at writing papers than most students” LLMs sure did pretty bad at those tasks over multiple rounds.

  • source
  • parent
  • hideshow 3 child comments
  • [–] a comment that isn't shown here: deleted, from someone you've blocked, or on another page. view the thread from it
  • [–] 6 points 6 months ago* (1 child)

    What the fuck? The only task that didn’t degrade across most models was Python. Very basic things like JSON, Makefiles, and schemas got screwed. Fiction, emails, and food menus got screwed. Did you even bother to read the legend? If you consider a single pass to be “producing and manipulating language” you didn’t bother to read the idiotic article you started this thread in support of. Good luck.

    Edit: why do you lie?

    Catastrophic corruption (80 and below) occurs in more than 80% of model, domain combinations.

  • source
  • parent
  • hideshow 1 child comment
  • load more comments (1 reply)
  • load more comments (1 reply)
  • load more comments (1 reply)
  • [+] [S] 6 points 6 months ago* (last edited 6 months ago) (2 children)
  • [–] 8 points 6 months ago (1 child)

    Seems like an "evil" and dangerous talking point. To me, the value of consciousness isn't in ita evolutionary efficiency.

    It's not a question of the value of consciousness, it's a question of its necessity. If an unconscious "zombie" can be, to an external observer, indistinguishable from a conscious being, then that means we've been overestimating the importance of consciousness for intelligence. Like Dawkins says in the article, there could be unconscious aliens out there who are nonetheless as intelligent as (or more intelligent than) humans. This isn't a new concept -- it's been explored many times in scifi -- but AI is now bringing the question from the realm of philosophy to the real world.

    I know people working in AI insist otherwise but I see talking with LLM not as them thinking, but as them selecting the right combination of data that correctly continues a conversation.

    This is less true than it ever was with reasoning models. Some of the latest reasoning models don't necessarily even reason in English anymore but rather an eclectic mix of languages. The next step after that is probably going to be running the reasoning in latent space (see e.g. Coconut), which basically means the model skips the language generation layer altogether and feeds lower-level state back into itself. Basically it is getting closer and closer to what most humans consider "thinking".

    But even besides reasoning models, I believe LLMs aren't as different from human language production as many people think. The human speech centre, in a way, also just selects the right combination of data to continue a conversation. It frequently even hallucinates (we call this "speaking before thinking") and makes stupid mistakes (we provoke these with trick questions like those on the Cognitive Reflection Test). There's also some fascinating experiments in people who have had the connection between their brain hemispheres severed that really suggest our speech centre is just making things up as it goes along.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 3 points 6 months ago (9 children)

    LLMs are able to do things we previously thought only conscious beings would be capable of doing

    "We" as in lay misunderstanding of some pop science, still don't get what consciousness is and can't describe it. There are people alive today who didn't believe in their youth that black people are fully conscious, Dawkins demonstrated by his communication to his personal friend and hero Epstein, that he doesn't fully believes that women are conscious. What we thought or didn't think of previously can't be a good indication of anything.

  • source
  • parent
  • hideshow 9 child comments
  • [–] a comment that isn't shown here: deleted, from someone you've blocked, or on another page. view the thread from it
  • [–] 1 point 6 months ago (3 children)

    That 20% figure is just a clear indication how shit people are at conducting such a test, and that was basically my original point. 2 in 10 times people were convinced by a particularly echoey room.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 1 point 6 months ago* (last edited 6 months ago) (2 children)

    Turing test can be reliably passed by a bot that repeats last part of the previous sentence with a question mark at the end [...]

    If an LLM is correct 2 in 10 times, would you call it "reliably correct"?

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 6 months ago (1 child)

    If a person murders people only two days out of 10, they're a murderer, in order to not be a murderer they need to never do that.
    Reliably correct is when you're correct always. Demonstrably incorrect is when you're incorrect even sometimes.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 6 months ago* (last edited 6 months ago)

    Reliably correct is when you're correct always.

    Agreed, except I add "almost". "My car reliably starts" it starts "almost always": more than 2 in 10 times. "You reliably turn up on time" doesn't mean you're late 8 in 10 times, it means you almost always turn up on time. To "almost always", or "reliably" a thing: it means you fail 1 in 100, in a 1000, in 10,000 times. 10k is hyperbole, but the idea is clear right? Almost always/reliably != failing 8 out of 10 times.

    Your original point that these bots, that pass 2 in 10 times, reliably pass was wrong. Because: they dont "always pass", they don't "almost always" pass, they dont, even "pass in the majority of times", they rarely pass.

    Let's add our reliable = always substitution to the quote:

    Turing test can be [always] passed by a bot that repeats last part of the previous sentence with a question mark at the end [...]

    You see how that's wrong not just in fact, but in spirit too?

    If a person murders people only two days out of 10, they're a murderer, in order to not be a murderer they need to never do that.

    Relevance? Who says "Fegenerate is reliably a murder?"

    Demonstrably incorrect is when you're incorrect even sometimes.

    Relevance? You didn't use the word "demonstrably passed'. I'd have no problems is you did?

  • source
  • parent
  • load more comments (5 replies)
  • [–] 2 points 6 months ago (32 children)

    As LLMs have developed and have been able to cram more and more "thoughtlike" behaviour into smaller RAM and less computation, I've steadily become less impressed with human brains. It seems like the bits we think most highly of are probably just minor add-ons to stuff that's otherwise dedicated to running our big complicated bodies in a big complicated physics environment. If all you want to have is the part that philosophizes and solves abstract problems and whatnot then you may not actually need all that much horsepower.

    I'm thinking consciousness might also turn out to be something pretty simple. Assuming consciousness is even a particular "thing" in the first place and not just a side effect of being able to predict how other people will behave.

  • source
  • parent
  • hideshow 32 child comments
  • [–] 10 points 6 months ago* (last edited 6 months ago) (27 children)

    Brains aren’t impressive because of their compute (which is both immense and absurdly efficient) or their ability to predict the future (technically the main function of evolved minds). They’re impressive because they’re conscious. The fact that organic brains can also engage in hierarchical abstraction, which no digital computer (or Turing machine) can do by definition, is icing on the cake.

    (The halting problem and Godel’s incompleteness and Traski’s undefinability theorems all seem to suggest that analog, not digital computation is more likely to be involved in consciousness, if at all.)

  • source
  • parent
  • hideshow 27 child comments
  • [–] 3 points 6 months ago (2 children)

    I don't see why there would be any fundamental difference between analog and digital computing. Digital computers can emulate analog computing, and I doubt consciousness arises from having theoretically infinite decimal precision, because in practice analog systems cannot use infinite precision either. Analogs (heh!) of the halting problem and the theorems you mention also exist for analog computing.

    Quantum effects in the brain are a slightly more plausible explanation for consciousness, but currently they teeter on magical thinking because we don't really know anything about what they would actually do in the brain. It becomes an "a wizard did it" explanation.

    So in the end, we just don't know.

  • source
  • parent
  • hideshow 2 child comments
  • [–] a comment that isn't shown here: deleted, from someone you've blocked, or on another page. view the thread from it
  • [–] 2 points 6 months ago

    I have a master's degree in computer science.

    Obviously I meant "I don't see why there would be any fundamental difference between analog and digital computing [when it comes to consciousness]."

  • source
  • parent
  • load more comments (1 reply)
  • [–] 5 points 6 months ago (4 children)

    You're going to have to do a lot more to justify the leap from Godel's Incompleteness and the Halting Problem to "digital is limited, analog is not", because neither of those things have anything to do with digital processes at all, and in fact both came about before we'd invented digital computers.

    To me this comment sounds like when popsci gets ahold of a few sciency words and suddenly decides everything is crystal vibrations universal harmonics string theory quantum tunneling aligning resonance with those around you.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 2 points 6 months ago* (last edited 6 months ago) (3 children)

    The situation is the following.

    1. Brains are analog computers, which are digitally irreducible.
    2. There are stringent limitations on Turing machines (digital computers),
    3. We can’t extract semantics from syntax, and so…

    We’ll probably need analog computation, currently in its infancy, to get artificial (inorganic) consciousness.

    I study metaethics and philosophy of mathematics. These problems are real, and I am being honest with you.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 1 point 6 months ago (2 children)

    That is not the situation. 😛

    Analog signals are not digitally irreducible without presuming there's no level of noise floor under which greater detail is irrelevant, Turing's machines are not digital by their construction and predate the concept by a long time, and the first computers we built were analog and we invented digital computers later because they were cheaper and more efficient and easier and more reliable.

    Also the halting problem doesn't say "there are things which a computer can't know but a human can", it says "there are some things that cannot be known".

    Similarly Gödel proved that there will always be true things about a system that cannot be proven from within the system, that is using its axioms. That was a real bummer for folks trying to prove all of math with a small set of axioms. But that does not mean there are things math can't know that humans magically can, it just means there's other math, outside the axioms, that are true without following from them, in math. He proved it with math, after all. It doesn't claim to give any special abilities to human brains.

    And also, again, nothing Gödel or Turing ever said has anything to do with the concept of "digital" anything. I think you're using the term "digital" to mean "rulesy"? Which is not even close to what it means?

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 6 months ago* (last edited 3 months ago) (1 child)

    Turing's machines are notdigital by their construction

    This isn’t even wrong.

    Go take a theoretical computer science course. Lectures from MIT and Carnegie Mellon are available on YouTube.

    Stop watching podcasts with pseudo-intellectual media grifters and read the actual research literature by real philosophers and mathematicians on these otherwise arcane topics.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 2 points 6 months ago (9 children)

    (The halting problem and Godel’s incompleteness and Traski’s undefinability theorems all seem to suggest that analog, not digital computing is responsible for consciousness.)

    I hear that argument from time to time, and I never found a source for it. I want to understand the original claim. Because it doesn't make any sense when people bring it up. because both theorems do not have anything to do with the areas it's applied to. I understand why people think it does, but it just doesn't

  • source
  • parent
  • hideshow 9 child comments
  • [–] 2 points 6 months ago (8 children)

    The simplest way to understand this problem is as follows.

    1. Analog computation is not digitally reducible. (Brains are analog computers.)

    2. Turing’s infamous Halting Problem.

    I can write more about this and point you to more technical discussions if you want.

  • source
  • parent
  • hideshow 8 child comments
  • [–] a comment that isn't shown here: deleted, from someone you've blocked, or on another page. view the thread from it
  • [–] 1 point 6 months ago* (last edited 3 months ago) (4 children)

    You will find what I said in any philosophy of mathematics textbook dealing with the subject. In fact, I am paraphrasing the Oxford logician Joel David Hamkins.

    You’re welcome to also read Shapiro’s famous paper. These results have been well understood for half a century, although because the implications are ultimately metaphysical and not mathematical, we can’t be sure of the wider consequences, if any.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 1 point 6 months ago (3 children)

    ah, now we're getting somewhere.

    Going through some of the related paper abstracts, including speculative comments by Gödel: this is pure philosophy. Nothing that is set in stone. Which now points me back to my initial statement, where we can discuss all we want, but in the end it's philosophy. Not "hard" ("provable") science

  • source
  • parent
  • hideshow 3 child comments
  • [–] 1 point 6 months ago* (last edited 3 months ago) (2 children)

    Here is what we know for sure:

    There can be no enumerable list of axioms for the true statements of mathematics. No computational procedure could exist to determine whether propositions are valid, provable, or even equivalent. And no matter how you formulate the number-theoretic axioms, a mathematician would always have insights (for instance, about whether a Diophantine equation has a solution) that are both clearly “true” and obviously unprovable. This holds true for all digital systems.

    Here is what we don’t know for sure:

    The metaphysical implications.

    Your distinction between science and philosophy is incorrect. Science is inductive and abductive. It can’t “prove” things deductively. That’s mathematics and philosophy.

    Philosophy also determines the formal systems we use as a basis of reasoning, for instance, in science.

  • source
  • parent
  • hideshow 2 child comments
  • [–] a comment that isn't shown here: deleted, from someone you've blocked, or on another page. view the thread from it
  • [–] 1 point 6 months ago*

    Yes, of course you can prove things in philosophy. Have you ever heard of syllogistic reasoning? The basis of… you know… proofs?

    All science is philosophy. Hence the P in PhD. Not all philosophy is science. Hope that helps.

  • source
  • parent
  • load more comments (1 reply)
  • load more comments (3 replies)
  • [–] 2 points 6 months ago* (8 children)

    I'm still awaiting a widely accepted method of actually measuring "consciousness." It's a conveniently nebulous property.

    And simply defining it as something computers can't do is even more convenient.

  • source
  • parent
  • hideshow 8 child comments
  • [–] 3 points 6 months ago* (7 children)

    That doesn’t change the fact that I am conscious.

    Also, I never said computers can’t be conscious. I said that digital computers (Turing machines) probably can’t. Quantum and analog computers have no such theoretical constraints and they’re far, far more prevalent given that they’re found in every living creature.

  • source
  • parent
  • hideshow 7 child comments
  • [–] a comment that isn't shown here: deleted, from someone you've blocked, or on another page. view the thread from it
  • [–] 4 points 6 months ago* (last edited 6 months ago) (5 children)

    This is called the problem of other minds. Of course I can’t be certain about the consciousness of others. I can only be certain about my own.

    We do have a way of measuring the correlates of consciousness. But we have no clue how to detect the presence of subjective experience using quantitative methods.

    Philosophy departments (which is where any discovery on this front will originate) are heavily defunded. If you’re waiting for physicists or biologists to figure this out you’ll be waiting even longer.

  • source
  • parent
  • hideshow 5 child comments
  • [–] a comment that isn't shown here: deleted, from someone you've blocked, or on another page. view the thread from it
  • [–] 2 points 6 months ago* (last edited 3 months ago) (3 children)

    It is not presumptuous at all. Inference to the best explanation is how you know (almost) anything.

    1. This table isn’t conscious.

    This is my justified belief. No inferential claim is guaranteed and all objective claims are inferential (which is why science isn’t absolute).

    That said, I have strong reasons to think that tables aren’t conscious. They might be, but I’m epistemically compelled to believe otherwise.

    1. ChatGPT isn’t conscious.

    Ditto. It would be irrational for me to believe otherwise given the strong evidence.

    That you “don’t know for sure” is an implied disclaimer for every scientific claim.

    If the evidence is ambiguous, we say so. Regarding ChatGPT, the evidence is unambiguous.

    1. I am conscious.

    This is a non-inferential claim that I know through direct contact with reality. It is a priori.

  • source
  • parent
  • hideshow 3 child comments
  • load more comments (3 replies)
  • load more comments (1 reply)
  • load more comments (1 reply)
  • [–] 2 points 6 months ago*

    I've steadily become less impressed with human brains.

    You need to lay off the AI if it's making you this weirdly misanthropic.

    This is how tech bros justify causing harm: they genuinely don't care, because they think of the un-"enlightened" as less worthy of existing

  • source
  • parent
  • [–] 1 point 6 months ago (2 children)

    If all you want to have is the part that philosophizes and solves abstract problems and whatnot then you may not actually need all that much horsepower.

    Just massive data centers requiring tons of energy and cooling, with a model developed by human brains and trained on all of human knowledge these developers can get their hands on, painstakingly labeled by vast teams of people so that the model can spit out seemingly correct answers.

    ls the AI actually philosophizing and solving abstract problems or is it merely regurgitating philosophies and solutions that exist within its training set?

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 6 months ago (1 child)

    It's actually solving abstract problems.

    Also, local models are available that are quite good and run on a standard consumer-grade GPU.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 6 months ago

    One problem and...

    Still, it required humans to apply the finishing touches.

    “The raw output of ChatGPT’s proof was actually quite poor. So it required an expert to kind of sift through and actually understand what it was trying to say,” Jared Lichtman, a mathematician at Stanford University whose doctoral thesis centered on one Erdős’s conjectures, told SciAm.

  • source
  • parent
  • Yeah i dont really believe in consciousness, it's the just the dynamic firing of neurons, it's an emergent trait in other words. It's like traffic, you will never find it if you zoom in to one car. You have to see it at a distance. Same with consciousness, if you zoom in it's not there anymore.

  • source
  • parent