I think the problem stems from how LLMs are marketed to, and perceived by the public. They are not marketed as: this is a specific application of this-or-that AI or ML technology. They are marketed as "WE HAVE AI NOW!", and the general public who is not familiar with AI/ML technologies equates this to AGI, because that's what they know from the movies. The promotional imagery that some of these companies put out, with humanoid robots that look like they came straight out of Ex Machina doesn't help either.

And sure enough, upon first contact, an LLM looks like a duck and quacks like a duck ... so people assume it is a duck, but they don't realize that it's a cardboard model of a duck with a taperecorder inside that plays back quacking sounds.