LLMs and generally deep learning as a whole have a number of serious limitations that would seemingly prevent AGI. That's not even getting into the fact that a lot of the fundamentals of LLMs havent changed since they were rolled out almost a decade ago, so it seems unlikely we struck gold in 2017 and just havent worked it out yet.
A great example is that LLMs can't do most math on their own. The Dag like structure of deep learning puts a finite limit on the number of calculations that are possible.