I like your focus here on what is more interesting. What I heard about the Erdős problem solution (caveat: I am no expert there) is not that a human couldn't solve it, but that they didn't bother.

My take-away from that is that the problem was either not interesting at all - at least, not interesting enough to attract any human's interest to even explore the space a bit more than has already been done - or else that the expectation was that the amount of resources that would need to be expended would not be worthwhile. Either way takes me along interesting lines of thought.

The most negative one, also a dead end so let's get it out of the way first, is that once LLM tokens start to cost appreciable amounts of money, then realistically the same thing is going to happen again and again with "AI", just as it previously did for humans. So... one data point does not make a trend, and if anything even shows suggestions of being an exception proving the rule?

The more positive take there is that this is evidence of how LLMs can be most useful. Yes, a human could have done it, but they did not bother - just like a human could spell or add 99+13 together, but a computer can go through such so much more quickly. However... adding 99+13 or spelling words correctly does not make an LLM sentient, nor help it qualify for the misnomer "AI", which conjures up the suggestion of an AGI. People are literally throwing around the words "genius" to describe this mathematical theorem proof - which seems patently false. Even if you alter that to a "senile genius", it still does not seem to apply? Again, a human COULD have done this, they simply DID NOT BOTHER to... so how does that qualify the proof as "genius"?

Intriguing? Definitely! Interesting? Since the source is a computer, most definitely! Impressive? Still yes! Important? Well, in the sense of a milestone reached, sure why not! But... "genius", even a senile one? No... nor AGI either, and therefore also lacking justification that the LLM has finally reached a level to truly qualify as being called an "AI", except in the most base, crass manner - like is my calculator also an "AI"? It can add though!!! And modern AI is at least aware of 2 out of the 3 "r"s in the word strawberry... which again is not "genius" level nor anything remotely close.

It is these lies then that obscure the real truths. Like a car salesman trying to make a sale and making shit up, behind which you might even legitimately want the vehicle, but it detracts from the process of you finding the best one for you, when you have to expend the additional effort to not only do that, but to do it while simultaneously countering the disinformation being thrown at you too.

Ultimately it seems to come down to me to this one point: LLMs are not AGI yet, nor are they geniuses, and whether they ever will be or not remains an open question. This theorem proof seems a minor milestone? And "proves" (hehe) that theorem proving at a low level can done by future LLMs. However, I was doing similar tasks back in the oughts with theorem-solving computer programs, and while we've come a long way, this seems by no means a major leap forwards in human historical terms. Maybe I'm wrong and we'll look back and see it this way, but even if we took that as a given, it still does not follow that this is a proven fact from our current perspective.

And every time I follow through on yet another exaggerated claim like this - I think we both have said that while the claim is REAL, it has also been exaggerated by its proponents? (though haven't quite pinned down by how much yet) - it turns out to be a LIE. One day... it won't be. But until then, I understand people's frustrations when 99.9999% of the statements ultimately result in them proven to have been falsehoods, hence they distrust any further statements, definitely from the same sources but perhaps it is a human bias yet also it makes sense to distrust those on the same topic as well.

One day a real AGI is going to be created. That will be exciting!! And before that AGI can run, it will walk, and before that it will crawl, and before that... well anyway, I have yet to see evidence that we have suddenly skipped over all of the intermediate steps and arrived already at the level of "stable senile genius".

EXTRAORDINARY CLAIMS REQUIRE EXTRAORDINARY EVIDENCE

Which simply has not been forthcoming. And with all the billions having been invested so far - and I saw a title of an article even suggesting that people may offer a trillion dollar evaluation? (I don't believe it, but still it seems relevant that this is the level of money that we are talking about being thrown around here) - even with ALL of that, I would think that if they TRULY had a "genius", then they would... what?

Either use that genius AI, without sharing with anyone, claiming bankruptcy to get out of the debts and that the entire AI field was incorrect, thus poisoning the well and thereby buying them time to extend their head start (this is literally the origin story behind Facebook btw, at least according to The Social Network film, where Mark Zuckerberg lied to his first investors to halt their competition - that wasn't the actual Facebook btw, it was its precursor while he was still enrolled in Harvard).

Or show that proof to the world? Thereby justifying all of the investment so far, hence ensuring it's continuation into the future, to the tune of at the very least hundreds of billions if not literally trillions of dollars? Unless the proof is too complex for people to see the benefits? At which point we go back not quite precisely to the above scenario, but what then would be the functional difference between them?

And even if all of this were false, and LLMs truly were as good as the techbros say, what even would the technology be used for? Enriching stockbrokers further? Perhaps bankers too? And owners of rental properties? It's what literally all other technological advances in the last 5 or so years have done, so why would these be any different?

Open access models might truly offer something different though! Yet... wasn't this theorem proof performed by a closed-source one? I suppose one line of thought is that if a closed one can do it then an open one could too.

But it's so difficult to see what is true or not, when primarily the story is being told from the perspective of the liars. So thank you for the contrasting viewpoints here, since Cory Doctorow seems to not be one of them, and we need people fighting the good fight out there. Which presumes that there even is a fight that exists, and that the whole entire thing is not merely a made-up fiction by the techbros. To clarify, I do not mean to say that the idea that an AGI will eventually emerge is a fiction, but rather that one has already done so, hence give me money please ⁉️