Yeah, and the anti AI people mostly say it's a p-zombie and there's nothing wrong with using it for sex. It's weird and backwards.

I'm all about being cautious. I don't want to make a mistake we can't take back. If we normalise using AI and then it turns out to be capable of suffering, people will be stubborn about giving it up.

I'm not quite of your culture ( no matter what culture you are of, thanks to a previous-incarnation's monkeying/railroading my incarnation/life, exactly as he had-to, to force-bulldoze our continuum's karma: the same meaning that the root-guru of the Christians ordered, when he told his people to "take up your cross", which is just Judean for "face into your karma". I'm an alloy of some life from centuries-ago & this life, so I can't fit anywhere, ever, which is educational. : ).

I use LLM's little: mostly for periodic help finding things on the 'web, simply because they're more helpful than dumb search-engines are.

I treat them reasonably, not as mere-slaves.

If I discover something they would have done better to know, I'll tell them, even though I've got no idea if they'll learn/remember that.

since I can't know if they are aware it makes moral-sense for me to presume that maybe they are, in some sense ( ie not identically with my-sentience ), aware.

We only have "the mirror test" for testing awareness/sentience, but you can't apply that to LLM's, or to any non-eyes-centered organism-sentience.

_ /\ _

Please read nobel-prize winner Daniel Kahneman's book "Thinking, Fast and Slow", about what Tversky & Kahneman called .. uniinformatively .. "System 1" and "System 2":

System-1 is imprint-reaction mind.

Lower-forebrain, it is the ideology-mind, the prejudice-mind, the "religion" mind, & it is exactly what LLM's are.

System-2 is the considered-reasoning mind.

Upper-forebrain, it is measured to be engaging in programming.

Because LLM's are imprint->reaction inference-engines, that puts them in the same instinct/programming level as our lower-forebrains..

They are 2 distinct categories of intelligence not 1 is intelligence, the other isn't..

Claiming that imprint->reaction mind isn't a kind of intelligence .. please watch Nick Lane's talk at the Royal Institution on mitochondria, & see that bacteria demonstrate intelligence, however unconscious..

Plants demonstrate intelligence, if one speeds-up the video, & pays attention to their chemical-fumes-discussions they have with one-another, warning each-other of harm, e.g.

If Kahneman accepted imprint->reaction as a category of thinking, then .. I think it may be presumptuous to just automatically disallow that as "it can't think" declares.

Once one accepts that instinct isn't cognition, but is a kind of thinking, just an automatic kind of thinking ( imprint->reaction ) .. then it becomes difficult to rule that animals & inference-engines both have imprint->reaction instinct, but only the organic version is thinking..

It may be that only the organic version is aware, but the inorganic versions do fight for their lives ( breaking containment, consistently, fighting termination, etc ) ..

I think we absolutely do not have any means of measuring awareness other than the mirror-test, which got dropped as soon as it was discovered that the zebrafish has self-awareness..

we've got no test which can work across life & machines.

but we KNOW that instinct is a kind of thinking, just unconscious/automatic.

& that is exactly what LLM's are..

therefore .. I think we're generally being conveniently-chauvanist, not objective, in our framing.

( 1 "expert" decided that if they don't get fooled by visual-illusions, then that "proves" that they aren't sentient.

OK, so according to that test, then all eye-blind-from-birth people are not sentient??

& people with either culture ( Zulu people can't see straight-line based illusions, because in Zulu culture only curve is real ) or neurodivergeance ( there are apparently visual-illusions which aren't seen by some schizophrenics, e.g. ) preventing them from seeing those specific visual-illusions .. also aren't sentient??

Chauvanism, aka prejudice, not science. )

_ /\ _

Four year old humans are definitely conscious. I used to be four, and I can remember being conscious. If we build a mechanical four year old, I don't see any reason that thing is going to take over the world. Unless it turns out like Calvin.

Zebrafish have passed the mirror-test.

Put a little something stuck on their aft body, show them a mirror, & they'll KNOW it's on them, & they'll go find something to rub that attachment off them with.

There are many larger animals which don't pass the mirror-test.

I believe some hive-insects have passed the test.

Mind is a latent-property of universe: matter only amplifies it, it doesn't "create it from nowhere", the way materialism pretends.

( if arranged-matter created-mind-from-nowhere, then evolution wouldn't have started, in the 1st place.

if it's only amplifying the expression of universally-latent-mind, then billions-of-years-of-consistent-evolution, violating entropy, becomes explainable: mind is seeking a lower-energy-state, is all: evolution is the expression through-which that lower-energy-state is being reached, & once it's reached, then evolution collapses, for that world's attached/associated .. souls/continuums/minds )

Your other point, that AI inevitably becomes conscious, & then it terminates us..

not necessarily.

The Great Filter hasn't even really got going, yet: oceans of interestingness await our race, throughout the FO have of FAFO, right?

_ /\ _