They also say that 91% of answers contained at least some issues. That's pretty bad.

Perplexity cited at least one BBC source in all responses, ChatGPT and Copilot in 70% of responses, and Gemini in 53% of responses.

I’m thinking there are additional issues here besides AI just making errors. I’m curious what their prompts/instructions were because they clearly weren’t limiting the tools to only using BBC sources. Because of that, the tools could be citing sources that are incorrect, possibly even AI generated.

In December 2024, the BBC carried out research into the accuracy of four prominent AI assistants that can search the internet – OpenAI’s ChatGPT; Microsoft’s Copilot; Google’s Gemini; and Perplexity. We did this by reviewing responses from the AI assistants to 100 questions about the news, asking AI assistants to use BBC News sources where possible.

The answers produced by the AI assistants contained significant inaccuracies and distorted content from the BBC. In particular: …

51 % significant issues, 19 % factual errors, 13 % altered or invalid quote citations

It's using data over a year old.

Some of the examples seem like nothingburgers, the Lucy Letby example actually looks like the AI failing "safe" by refusing to make a conclusion.

(It's also questionable of the BBC to insist that de jure and de facto innocence is the same thing, it probably is in the Letby case, but you wouldn't want it to necessarily make that assumption in, for instance, the OJ Simpson case)