Bookmarked, looks like an interesting read, thanks.

Some of this may come as news to a lot of the machine learning community

Does it? I only have pretty basic knowledge in the ML field, from like two courses during my Masters in gamedev around 8 years ago, and I though that it's a basic fact of most of the ML algorithms, that simply throwing more data at it won't get it "smarter", as in from the basic understanding of how ML works, it's pretty apparent that you can't get anything like an AGI with the current algorithms.

You're basically just approximating a function (which is my understanding of what ML does) of what's the next word based on previous senteces, your dataset. It kind of makes sense it would converge into absolute mediocrity (not even mediocity, because a lot of data in the datasets is very probably wrong), and not be able to come up with new things.

But, we've never really learned about transformers, since that tech wasn't yet part of our syllabus, so I might be wrong/overly simplyfing things.

Wasted or stolen? Because somehow, despite this "mistake" it seems like some people are still getting much richer. Funny how that works.

That's called a bubble. Hype it, invest in it, loose it.