I think arguments like this are misleading when discussing the question of AI possessing consciousness. Materialism, dualism, panpsychism, idealism, etc are all philosophical concepts; they attempt to answer “what is the nature of consciousness and how does it arise from the body?” from a philosophical standpoint. All of the various options have pros and cons and there is no clear “winner.”
However, there are decent scientific definitions for consciousness. These attempt to answer the question of “what do we mean when we say something is conscious?” from a scientific standpoint. There are criteria that neuroscientists, doctors, etc generally agree on.
So it’s not really correct to say “no one knows what consciousness is” when evaluating whether something is conscious. And I think it’s quite clear that current LLM algorithms could never be conscious, using the current accepted scientific criteria.
(I’m not a philosopher or a neuroscientist, but I do have a degree in computer science and have a solid understanding of how LLMs work.)