Uh, I'm really unsure about the engineering task of a few years, if the solution is quantum computers. As of today, they're fairly small. And scaling them to a usable size is the next science-fiction task. The groundworks hadn't been done yet and to my knowledge it's still totally unclear whether quantum computers can even be built at that scale. But sure, if humanity develops vastly superior computers, a lot of tasks are going to get easier and more approachable.
The stochastical parrot argument is nonsense IMO. Maths is just a method. Our brains and entire physics abide by math. And sure, AI is maths as well with the difference that we invented it. But I don't think it tells us anything.
And with the goal, I think that's about how AlphaGo has the goal to win Go tournaments. The hypothetical paperclip-maximizer has the goal of maximizing the paperclip production... And an LLM doesn't really have any real-world goal. It just generates a next token so it looks like legible text. And then we embed it into some pipeline but it wasn't ever trained to achieve the thing we use it for, whatever it might be. That's just a happy accident if a task can be achieved by clever mimickry, and a prompt which simply tells it - pretend you're good at XY.
I think it'd probably be better if a customer service bot was trained to want to provide good support. Or a chatbot like ChatGPT to give factual answers. But that's not what we do. It's not designed to do that.
I guess you're right. Many aspects of AI boil down to how much compute we have available. And generalization and extrapolating past their training datasets has always been an issue with AI. They're mainly good at interpolating, but we want them to do both. I need to learn a bit more about neural networks. I'm not sure where the limitations are. You said it's a practical constrain. But is that really true for all neural networks? It sure is for LLMs and transformer models because they need terabytes of text being fed in on training, and that's prohibitively expensive. But I suppose that's mainly due to their architecture?! I mean backpropagation and all the maths required to modify the model weights is some extra work. But does it have to be so much that we just can't do it while deployed with any neural networks?