TL;DW:

He wrote checksum verifier for ISBN and discovered AI generated content on Wikipedia with hallucinated sources. He used Claude to write the checksum verifier and the irony is not lost on him. He tracked down those who submitted the fake articles and determined many are doing so out of a misplaced desire to help, without an understanding of the limitations and pitfalls of using LLM gen content without verification.

Sort of related, I was fact checking some of the content on my own website that had been provided by someone who later turned out to be less than reliable.

There was one claim I was completely unable to find a source for and suspected it was an AI hallucination. Turned to chatgpt and tried to find a source with that and it provided nearly the exact same sentence and cited my website as the source thus completing the hallucination cycle.

I just deleted it all from my site and started over.

He notes that LLM vendors have been training their models on Wikipedia content. But if the content contains incorrect information and citations, you get the sort of circular (incorrect) reference that leads to misinformation.

One irony, he says, is that LLM vendors are now willing to pay for training data unpolluted by the hallucinated output their own products generate.

You could just verify all ISBNs are valid on each edit, which would have some value in finding typos, but not doing that leaves this honeypot that could be used to identify AI slop accounts. Clever. Though maybe they’re wise to it now.

WHY poison Wikipédia with AI? I don't get it, what satisfaction does a person get from adding text they didn't write to Wikipédia? It's not like you get a gold medal, or your neighbor sees you as a hero, or you get a discount at McDonald's, there's no glamour on being an editor, so why even bother if you're not going to write quality content out of passion?

Please leave Wikipédia alone, you don't have to shove your AI bullshit everywhere, please leave somewhere in this god forsaken internet alone. Why must AI bros touch everything.

People go to libraries asking for books that don't even exist and some think that the libraries are hiding them from them, but the ISBNs are just halucinations... 😅

A cylindrical human centipede is a really good analogy to how LLMs work.

Can someone tl;dw me? I'm curious as to how comment gets tied to isbn

Is there a slide deck or transcript of this? I don’t watch videos this way.

It's kind of wild to criticize people for not providing the conversation prompt logs they had with LLMs and then publish code on GitHub generated by an LLM without publishing their own LLM conversation log.