Are we not just prediction systems with more neurons?
Now im not saying LLMs are human at all because to be human is to have gone through our exact evolutionary path, but we really are in a lot of ways much more similar than you think. At least the speaking part of our brain is.
The argument that LLMs are fixed in time does not contradict consciousness. Say a human was stuck in a time loop and the human is asked the same question every single loop, at the exact same time, in the exact same way, with every single atom being in the exact place. Would their answer change? I would argue no, the same exact neurons would be stimulated every time that would cause the same exact answer every time.
We don't even have a solid definition of what is concious. Are insects conscious? Like the fruit fly, we have modeled all of its neural paths and it simplifies down to essentially being a robot that responds to stimuli.
What does make humans special is our ability to learn. But what when an LLM trains its self? Is that not learning? I think the question of consciousness is too complex to simply say one way or the other and it is foolish to simply it this way.