this post was submitted on 09 Jun 2026

235 points (98.0% liked)

Programming

programming@programming.dev

29,187 readers

2,780 active

102 users here now

Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!

Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.

Hope you enjoy the instance!

Rules

Rules

Follow the programming.dev instance rules
Keep content related to programming in some way
If you're posting long videos try to add in some form of tldr for those who don't want to watch videos

Wormhole

Follow the wormhole through a path of communities !webdev@programming.dev

founded 3 years ago

moderators

snowe@programming.dev

Ategon@programming.dev

UlrikHD@programming.dev

bugsmith@programming.dev

Spyro@programming.dev

LLMs are eroding my software engineering career and I don't know what to do (human-in-the-loop.bearblog.dev)

submitted 5 months ago by codeinabox@programming.dev to c/programming@programming.dev

81 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] sentient_loom@sh.itjust.works 2 points 5 months ago (20 children)

Nothing's inevitable. And as for "building a mind", while it depends on precisely what you mean by "mind", it's totally possible that only a biological brain can produce minds as we understand the word "mind". Building AGI doesn't necessarily mean building a mind. And since thoughts seem to be properties of "matter", and there seem to be rules about which configurations of matter produce mind, we don't necessarily know that there are other configurations that can produce minds. We might produce something else equally interesting which still is not a mind.

parent

hideshow 20 child comments

[–] mindbleach@sh.itjust.works 0 points 5 months ago (19 children)

it’s totally possible that only a biological brain can produce minds as we understand the word “mind”.

Bollocks. Thought is a process, like math. Nothing meat does with signals is impossible in other substrates.

At the utmost extreme: surely we can simulate physics at whatever level is necessary for virtual brains to function. Physical neurons are not gonna rely on quantum chromodynamics. Mere chemistry will probably suffice.

And hand-waving things that are like-minds-but sounds like Chinese Room nonsense.

parent

hideshow 19 child comments

[–] dave@feddit.uk 3 points 5 months ago (2 children)

You say that, and GAs were used decades ago to design FPGAs to a spec. The evolved design worked perfectly on the test chip, so the design was copied onto a second chip and it failed. The logic gates were identical but the GA had utilised microscopic differences in the substrate and there were large areas of programmed chip totally unconnected to the main circuit. Without them, the first chip didn’t work any more.

There are likely quantum effects available at the size / scale of neurons, and it’s brave to say evolution wouldn’t exploit them if there was some benefit.

parent

hideshow 2 child comments

[–] mindbleach@sh.itjust.works 1 point 5 months ago*

Yeah yeah yeah, probably exploiting capacitance instead of on-spec functionality, I'm well familiar with this example. It's not relevant - there's eight billion human brains in the world, and they generally still function despite the wild shit we put them through. They are not fragile.

A human mind is not balanced on a knife-edge, where one tiny difference breaks everything. They're complex enough that sometimes blowing a railroad spike clean through just alters functionality. It's still a mind. Subatomic interactions surely cannot be crucial here.

And again, this is only the extreme example. Y'think all known laws of the universe are mandatory? Great, simulate those too. Same answer: meat has no monopoly on thought because metal can fake the meat. There is no philosophical basis for even suggesting AGI is impossible, unless you start talking about souls.

parent

[–] sukhmel@programming.dev 1 point 5 months ago

The scale of neurons is too big for quantum effects, but that's contemporary understanding that may change in the future. We're really far from understanding both what mind is and how to make one

parent

[–] sentient_loom@sh.itjust.works 1 point 5 months ago (15 children)

Thought is a process, like math.

You're making a baseless assumption about the inner being of every process. If you simulate physics then you're actually doing different physics, where the map is not the terrain. If the hardware is different then the inner being of the thing may very well be different.

You're actually displaying a lack of imagination here. You're not considering things other than consciousness. If you simulate the processes which on the surface resemble the processes that you see in the brain when observing from the outside, what you produce may be something equally interesting and yet totally different in-itself from subjective experience.

You don't know as much as you think you know.

parent

hideshow 15 child comments

[–] mindbleach@sh.itjust.works 1 point 5 months ago (14 children)

Woo-woo nonsense. Indefensible without invoking the supernatural.

Everyone you've ever met is a meat robot, and they run on physics. How they feel about it does not change that.

And I already said, we invented p-zombies. LLMs display intelligence without consciousness. Your hand-wavy what-if doesn't work as a gotcha because we're already there. Nonetheless - these shortfalls are a big fucking hint that magic isn't real.

parent

hideshow 14 child comments

[–] sentient_loom@sh.itjust.works 1 point 5 months ago (13 children)

they run on physics

that's precisely what I'm saying

LLMs display intelligence without consciousness

That's precisely what I'm saying

doesn’t work as a gotcha

Nobody said anything about a gotcha

magic isn’t real.

Nobody said anything about magic.

If consciousness is a physical process, then a different physical process (such as an intelligent process running on different hardware) cannot be assumed to produce the same result (the result of conscious experience).

parent

hideshow 13 child comments

[–] mindbleach@sh.itjust.works 1 point 5 months ago (12 children)

Like planes don't experience flight unless they flap.

This is stupid. I acknowledge that's not an airtight logical counterargument, but just, come the fuck on. You are asserting that neurons made of silicon, with identical observable function, wouldn't count somehow. Charitably: wouldn't work, somehow. That at least distinguishes it from standard Chinese Room horseshit. But if we can fake every neuron to do the same thing, or simulate the entire physical environment to do the same thing, of fucking course it's going to do the same thing. If the laws of the universe somehow mean only meat can experience being a true Scotsman, we can fake those laws.

You've picked a philosophical nit that is somehow at odds with Turing completeness. Unless you think physics are incomputable - it cannot matter what substrate they run on. It's literally math.

parent

hideshow 12 child comments

[–] sentient_loom@sh.itjust.works 1 point 5 months ago (11 children)

Like planes don’t experience flight unless they flap.

Are you claiming that planes experience flight?

This is stupid.

k

come the fuck on

k

neurons made of silicon

Two problems with this: (1) The virtual neural net of LLMs don't have neurons made of silicon. Their neurons are virtual, abstract, not physical phenomena. (2) Even if we move to the idea of a positronic brain like Data from Star Trek or the Terminator, it still isn't our chemical-electrical brain which has different physical properties. This is very simple. It is a different physical object. It is different. It is not what we are.

Chinese Room horseshit

k

simulate the entire physical environment to do the same thing

If you are simulating it, it is a different thing.

It’s literally math.

This is a metaphysical assumption much closer to the "woo" that you keep accusing me of, and cursing at me about.

it cannot matter what substrate they run on

This contradicts your statement that "It's literally math" because you can calculate the difference between substrates.

parent

hideshow 11 child comments

[–] mindbleach@sh.itjust.works 1 point 5 months ago* (10 children)

If you are simulating it, it is a different thing.

No.

If a simulated person experiences consciousness, that experience is real. That's qualia, numbnuts. That is a mind.

I'm not gonna pick apart the word salad of 'calculating the difference between substrates' if you can't even keep your philosophy straight.

parent

hideshow 10 child comments

[–] sentient_loom@sh.itjust.works 0 points 5 months ago (9 children)

numbnuts

k

If a simulated person experiences consciousness, that experience is real.

You're just throwing the assumption of experience into the sentence for no reason. You have beliefs about this stuff, not facts.

parent

hideshow 9 child comments

[–] mindbleach@sh.itjust.works 1 point 5 months ago (8 children)

Do you know what if means?

You want to make this a matter of philosophy, and then you suck at philosophy. Hey buddy, do you have facts about other real people's experiences, or do you just have beliefs? Could you even demonstrate your own conscious experience to me?

And all of this is such tired Philosophy 101 crap, just so you can cling to 'aha but what if,' even though I have a concrete answer for what-if. Are we ruling out magic? Great, then physics can be simulated and a computer can host a mind that way. Its experiences would be identical to any free-range meatbag. If it wrote a book, you could read it. That would be real art. So in what fucking manner is its consciousness not real experience?

parent

hideshow 8 child comments

[–] sentient_loom@sh.itjust.works 1 point 5 months ago (7 children)

Hey buddy

Hello.

you suck at philosophy

When you said, "If a simulated person experiences consciousness, that experience is real", you're just saying "If my statement is true then my statement is true." You offer no basis for assuming that "a simulated person experiences consciousness" in the first place. You are simply assuming it. Your whole side of this entire conversation is just an assumption.

physics can be simulated

Yes and the simulation is a different physical process than the process that it's simulating.

parent

hideshow 7 child comments

[–] mindbleach@sh.itjust.works 1 point 5 months ago (6 children)

'If it happens, it counts' is not a tautology when you insist happening and counting are different things. To the guy in the simulation, any experience is real. His consciousness entails all the processes you insist must be accounted for. It works the exact same way as it does in real life with real meat.

If you would insist 'well that's only simulating consciousness' - that counts.

parent

hideshow 6 child comments

[–] sentient_loom@sh.itjust.works 1 point 5 months ago (5 children)

To the guy in the simulation, any experience is real.

Again, you have no basis for claiming that there is experience in the simulation in the first place. If you set up some simulation to mimic human behavior there's no reason to assume that it is experiencing that behavior. You're putting an assumption here.

If you would insist ‘well that’s only simulating consciousness’ - that counts.

I never insisted that it's "only simulating consciousness." I'm rejecting the baseless assertion that consciousness is what's being simulated. If you simulate the physical processes that we associate with consciousness (neural networks), even if it produces enough observable behaviors that we find the model useful, it is still a different physical thing and therefore we are unable to assert that it is experiencing consciousness.

parent

hideshow 5 child comments

[–] mindbleach@sh.itjust.works 1 point 5 months ago* (4 children)

Simulating physics from first principles is not "mimicking human behavior." Don't dismissively phrase it like a chatbot. If you insist the exact molecular exchanges in human neurons are a mandatory component, you could observe every subatomic event, not just the fact they talk to you like any other meatbag on the street.

I never insisted that it’s “only simulating consciousness.”

You just did! Again! You think an entire simulated human being, that acts exactly like a living person for the same underlying reasons, must be different - somehow. Your consciousness only arises from the laws of physics and the shape of matter, but if we simulated both of those exactly, and got indistinguishable results, then nuh-uh.

This is dualism. This is Descartes torturing dogs and insisting they only act like they feel pain, unlike us real humans, because we're different and special. It's straight-up Chinese Room horseshit, where no demonstrable evidence of conscious thought is enough, unless it fits your preconceived notions of what minds look like.

parent

hideshow 4 child comments

[–] sentient_loom@sh.itjust.works 1 point 5 months ago (3 children)

Simulating physics from first principles is not “mimicking human behavior.”

Our "first principles" are knowledge, not a new universe. You're not absolutely re-creating the entirety of reality. You're making a model that is useful, something that is "good enough" that it will produce outputs that can be used to roughly predict the outcome of real processes.

You think an entire simulated human being, that acts exactly like a living person for the same underlying reasons, must be different - somehow

No matter how detailed your simulation is, it's still physically different. You're admitting this every time you call it a simulation. You can tell the difference between a simulation and the thing it's simulating.

This is dualism

No, because I'm saying that consciousness is a physical process. You're saying that it's a mathematical process and that the substrate doesn't matter. I'm saying the substrate is a physical thing, and that consciousness is a physical processes, and so different substrates enable different physical processes. I don't claim that consciousness comes from some Beyond, or from Heaven, or from God. I'm saying it's a physical process, and that a simulation is a different physical process, no matter how detailed.

parent

hideshow 3 child comments

[–] mindbleach@sh.itjust.works 1 point 5 months ago (2 children)

You can tell the difference between a simulation and the thing it’s simulating.

The premise is - you fucking can't. It acts ex-act-ly like the real thing, and for all the same reasons. Given that unreasonably high standard, you still insist, nuh uh. Would never be conscious. We're beyond 'but what if.' You're explicitly arguing that machine consciousness would not count. That any difference from exactly how humans work cannot be a mind. Fuck off. That's just novel bigotry. Dog-torturing prejudice.

If you can have a long-ass argument with someone and go away figuring they're a person like you - that's consciousness. That's the only way you can judge the inner life of any person you have ever met. And you want to pretend that someone meeting that standard, while you observe their brain at a subatomic level, is disqualified? Come back and fuck off again.

You cannot laud the perfect exactitude of... squishy biology and quantum foam... and still say nuh-uh when a whole-ass person arises from those exact processes. If you mean anything when you say it's all just physics, then whatever physics are required for consciousness, can be faked. The resulting person really does think, as surely as an emulated calculator does real math. It's not simulating math. It's doing math. The device is simulated, but the answers are real. Get it?

parent

hideshow 2 child comments

[–] sentient_loom@sh.itjust.works 1 point 4 months ago (1 child)

The premise is - you fucking can’t. It acts ex-act-ly like the real thing, and for all the same reasons

You literally made the simulation, so yes you can tell the difference. And the math is inevitably different, because the math of the simulation includes the math which defines the different substrate. So it is physically different, a different thing. Assume whatever you want, but in the end it is a physically different thing, and it takes different math to fully describe it. We only know for sure that conscious experience happens with this substrate.

Fuck off

k

Dog-torturing prejudice.

k

fuck off again

good bye

parent

hideshow 1 child comment

[–] mindbleach@sh.itjust.works 1 point 4 months ago

the math of the simulation includes the math which defines the different substrate

2+2 on an emulated calculator gets a different answer than 2+2 on a real one, because there's different math. Math being physics in this case but not when it undercuts your whole point. Conscious experience must only be possible in this substrate, despite zero evidence to the contrary... and faking the entire substrate with physics math doesn't count because numbers work differently in rocks than in meat.

k.

If telling you to shut up is redundant then good riddance.

parent