If you feed them the approach the mathematicians were using, they for sure can solve it! It has never been published what source material their LLM was provided with. This is the equivalent of someone telling you a result, but never telling you how they got the information needed to achieve it in the first place.
There is value in this technology. But we should do out best to not conflate remixing shit with intelligence - these things are not the same at all. LLMs are more like the proverbial infinite room of monkeys with typewriters, with the very important restriction that these monkeys can never type something that resembles an original thought.