only "over 10 percent"??

how high could that number be and still make a valid, accurate statement? 25? 50?

I'm surprised it's only 10%. I remember a couple of years ago I worked on a team trying to build an AI tool that, as a feature, could cite sources. I was skeptical then because gemini couldn't reliably cite correct sources on a google search, so what made our team so confident the OpenAI api version could do that? I mean, it still can't because it doesn't know where the information is coming from, and I'm sure getting it to double and triple check its work to get it right leads to it getting stuck in cycles.

The point is you need to treat AI "facts" as if your trivia buddy told you a fun fact. It sounds right and you trust that they looked it up at some point, but they're confidently incorrect at least a couple of times a trivia night so you shouldn't put money on it without fact checking yourself.

"The lying bullshit machine keeps outputting bullshit and lies, engineers stunned"

To test whether the software accurately reported what its sources said, the researchers broke the overviews down into 98,020 individual factual claims. They built an automated verification pipeline, powered by a separate large language model, that read the text of the cited websites.

Uhh...

Maybe you shouldn't have the hallucination machine verify the output of the hallucination machine?

@remington Best to set homepage to udm14.com/ to avoid all that stuff.