Why do we experience things? Like, what’s the point? [...] it’s a byproduct of thinking in general.
I think so, too. It's a byproduct. And we're not even sure what it means, not even for humans. And there's weird quirks in it. When they look at the brain, the thought and decision processes don't really align with how we perceive them internally.
There's an obvious reason, though. We developed advanced model-building organs because that gave us an evolutionary advantage. And there's a good reason for animals to have (sometimes strong) urges. They need to procreate. Not get eaten by a bear and not fall off a cliff. Some animals (like us) live in groups. So we get things like empathy as well because it's advantageous for us. Some things are built in for a long time already, some are super important, like eat and drink, not randomly die because you try stupid things. So it's embedded deep down inside of us. We don't need to reason if it's time to eat something. There's a much more primal instinct in you that makes you want to eat. You don't really need to waste higher cognitive functions on it. Same goes for suffering. You better avoid that, it's a disadvantage almost 100% of the time. That's why nature gave you a shortcut to perceive it in a very direct way. No matter if you paid attention, or had the capacity for a long, elaborate, logic reasoning process.
That's why we have these things. And what they're good for. I don't think anyone knows why it feels the way it does. But it's there nevertheless.
They’re [LLMs] more like us than they’re like a calculator.
Now tell me why does an LLM need a feeling of thirst or hunger, if it doesn't have a mouth? What would ChatGPT need suffering and a feeling of bodily harm for, if it doesn't have a body, can't be eaten by a bear or fall off a cliff? Or need to be afraid of hitting its thumb with the hammer? It just can't. An LLM is 99% like a calculator. It has the same interface, buttons and a screen. If we're speaking of computers, it even lives inside of the same body as a calculator. And it's maybe 0.1% like an animal?!
If it developed a sense of thirst, or experience of pain, just from reading human text. That'd nicely fit the p-zombie situation.
It’s the right thing to do.
Yeah, I'm not sure about that. Most you do is muddy the waters with a term that used to have a meaning. I see the parallel, there's some overlap with being a vegan for environmental reasons and declining AI for environmental reasons. Yet they're not the same. I think the whole suffering debate is a bit unfounded, but it'd be the same thing if true... And I do other things as well. I order "green" electricity, buy used products, try not to produce a lot of waste. I'm nice to people because it's the right thing to do. But we can't call all of that "veganism". That just garbles the meaning of the word and makes it mean anything and nothing.

