When most people use terms like next token prediction they’re often trying to use it with an implication that it’s a simple thing and nothing big or important could grow out of it.

I'm saying that as much as we dress it up and pretend it's a silver bullet that can cure all ills, it isn't. They can be very useful and powerful in specific instances, but this will not achieve AGI.

Over the last several years we’ve insisted that AI cannot possibly share the most important aspects of the human mind, but one by one everything we pointed to in order to differentiate ourselves was shown by research to have a strikingly similar allegory in modern AI.

Citation needed. AI is trained to pretend to be human and it's real good at that. It's why we say it's thinking when it fundamentally is not. It's anthropomorphization writ large. It's just very repetitive math. Maybe brains are too, but it hasn't reached our level. And it's wrong all the time, which is a bit of a problem.

You genuinely believe that the researchers under tight NDA in the frontier labs agree with you on AGI not being anywhere near reach? Or do you think you have a deeper understanding of the topic?

My degree and I do think that while I'm not a doctor (a mere masters), I know enough to look at what they're doing, how they're doing it, what they're promising, what they aren't actually delivering, what damage AI* is doing, and what it is under the hood to know that what's been promised cannot happen with things as they are and may work to kill us all along the way (ecologically speaking).

plenty of people to lord over who are grateful for the wonderful people who stepped in and helped save them

Pure Hollywood. If you think anyone could engineer something with that level of precision AND manage to somehow dominate the world after such wildly fictitious devastation, I have AGI to sell you.

It isn't complicated. It's the oldest thing in the world. Grifters found a shiny and they want to get as much money as they can get and run before the bill shows up.

In the same way it’s accurate to call the human brain a prediction machine

I have seen no evidience presented that the human brain is simply making statistical predictions.

This seems to be a relatively new thing some people are asserting, because they see the LLM behavior and have decided that absolutely it looks so human like it must act the same, and since we understand that LLM is fundamentally statistical prediction of a token, then obviously human consciousness must operate on the same principle.

Even the latest models generate the sort of mistakes that stem from the fundamental limitations. A lot has been done in some contexts for making that not matter so much (e.g. in software development, the strategy is that code may have some verifiable goal, and the models can let the mistakes fly, then take the feedback from the goal, and iterate more). So it still can be very useful, but clearly it isn't human like because of some of the completely dumb behaviors that result from it not actually thinking about it. It is at it's best when either mistakes don't matter (particularly fiction) or at least first mistakes don't matter and can be automatically reconciled with facts.

By nature, the meme examples will get better because everyone talks about it and suddenly having all that discussion in training data and in search results, the statistics fix. However the operating principle behind them remains the same.