Try posting this on !localllama@sh.itjust.works.

As it happens I'm working on a chessdb.nu, which is supposed to be a set of tools and skills for llms to annotate, even Socratically coach chess positions and games.

Two observations, 1) I think the post is really just trying to explain that the LLM is not limited to the next token in its planning. It can have an idea of where it is going all the way to the last period and token prediction is just the step by step to get there. Even more interesting is that the LLM can change plans midstream (cf: seahorse emoji). And 2) without going so far as saying it is existentially making choices, it is showing complex reasoning and planning which is a few steps above a Markov Chain.

As for myself, I like the analogy of french curves and splines being fixed and added to with the LLMs internal reasoning and then walked token by token. But LLM's don't actually have to be that continuous and smooth, although in fairness when it isn't, it's usually a failure mode.

But what do they predict?

But if you are even a little bit educated as to what the textbooks say about LLMs, they are exactly next-token predictors. There's no ifs-or-buts about it.

Your chess analogy is misinformed too. Your description of neural networks trained on past master games reads like the machine is given pictures of the board and simply learns from that. It also assumes that is how modern day chess engines are written which is not completely true. It is called the concept of "Book" which records past moves in certain positions as a cache. And people don't call that "Next Token Prediction". It's not as sophisticated.. it's simply a lookup table or a cache.

Actually good Chess Engines are search engines walking over a tree or graph of possible next moves. One can argue this is actually a prediction engine... but "Prediction Engine" as a description feels like the machine is rolling the dice far more than a search.. which is the part that's misleading.

Here is my claim: LLMs as they stand today are fundamentally less sophisticated than a Chess Engine since the search part is absent from LLMs. This is not because LLMs are less powerful, no. Their domain is simply different from search engines. Token prediction IS the job description of an LLM. Which is why LLMs are not very good at chess.

That machine costs a whooping 6k€. That's nothing to sneeze at.

Ah I hadn't scrolled down far enough on the GH issue to see that caveat!

Had I made this I think I'd have not wanted to risk any accusations of hypocrisy! That said I reckon it's still in keeping with their point - i.e. LLM generated text can be helpful but not if you just copy/ paste it without attempting to comprehend/ integrate it.

So it's not written, but edited and extended by AI. So it... is mostly written by AI?

This is actually addressed in the footer!

Spellcheck/grammar is a different thing. The website is talking about generating the content with AI. I was responding to that.

Pick any of my longer responses here and there's a 90% chance that I copy-pasted it from an LLM. However, it doesn't read like an LLM and there's a good reason: I wrote it myself - AI only checked the grammar.

Just because AI was involved doesn't mean no effort was put into it. The opposite extreme of this are my illiterate friends who don't even try to write coherent sentences when responding to messages. They should use an LLM because forcing me to interpret their gibberish isn't particularly friendly either.

Agree with the concern, totally disagree with their answer.

Don't use the AI at all. Talk like a normal human. Dare to say you don't know. Or, you know, give a little of your time to care about someone else's questions enough to answer them yourself. Seriously, what's wrong with people that they think this is a remotely sane thing to do?

p.s. if you did this to me, and didn't stop when asked, I would stop talking to you. A friend doesn't do this.

That reads, ironically, very ai generated. Especially the other version on offer.

that site is very sensational. at least in the body of their article they provide some honesty. it gave the same advice when you say you are alone with someone from google. sure it needs to be fixed but there was sone cherry picking going to on the boost the sensational title.

This just in: Showing up uninvited, shitting all over everything, and making life empirically worse for everyone is generally frowned upon.

What a twist!

Maybe community-owned datacenters will, who knows.

Waiting for some Cortecs provider to make it available.... I use these small models usually without thinking. Don't feel like thinking helps a lot; if you need thinking maybe just use a slightly more powerful model IMO.

I kinda agree with his points that:

Currently, using local models on average consumer hardware is not really feasible. The math of building your own powerful lab to run these models just doesn't work out no matter what way you look at it... just look at PC component prices, factor in the electricity and so on....

People will generally go for the new shiny, powerful thing (that's what drives our economic system, after all).

Data center inference will always be more efficient.

I would also add that people are not tech savvy enough and or lazy to go through the hassle of setting up a local model.

So I do think running a model locally will continue being a niche unless some serious advances will be made in them.

It's important to note that even though running a local model might not make sense today, that doesn't mean we have to be stuck with large, proprietary USA models. Through something like Cortecs you can use small, open-weight models like GPT 20B, Gemma 4 26B, Qwen 9B and so on ... They're dirt cheap, some providers have environmentally clean data centers, strict privacy policy, etc.

Which is a great thing.

No it didnt. There was a big announcement post about this last week. Its all anyone is talking about.

you’ll still have AI like you do today

A great many people today use only the proprietary, software-as-a-service providers for their LLMs, diffusion models, etc., and those services are indisputably costing far more to provide, than the revenue gained from users. They are funded to ridiculous levels by investment from funders who haven't yet figured out they will not be getting paid.

That kind of service will not survive the bursting of the current hyperscaler bubble. We will not have today's low-priced access to (and therefore we will not have shoved down our throat, in every application and device and online service, at low cost) the gargantuan resource-guzzling data centres, because the hyperscalers will not be keeping those operating when their investment funds dry up.

But he can still write to stdout

Would be funny to see a version of this in which they’re trying not to use AI. I believe in the original they’re trying not to eat their cookies.

Intelligence is not the word I would use to describe the plagiarism bots

1.68 TB recommended RAM 😅

7/27*