Wanting a LLM do not hallucinate is like wanting a heater to not generate heat.

It's worth noting that humans aren't immune to the problem either. The real solution will be to have a system that can do reasoning and have a heuristic for figuring out what's likely a hallucination or not. The reason we're able to do that is because we interact with the outside world, and we get feedback when our internal model diverges from it that allows us to bring it in sync.

Hopefully there are people still working on non-llm type general AI, because i don't think we're ever going to get there with LLMs. The architecture just seems wrong to ever get there, and even Altman has said they probably can't solve hallucinations. We can probably go very far down this road and get them pretty good, but it's the wrong road if you want a real AI.

Then they will always be useless as standalone trustworthy agents.

The word "hallucination" itself is a marketing term. It's not because it's been frequently used in the technical literature that it is free of any problem. It's used because it highlights a problem (namely that some of the output of LLM are not factually correct) but the very name is wrong. Hallucination implies there is someone, perceiving and with a world model, who typically via heuristics (for efficient interfaces like Donald Hoffman suggests) do so incorrectly leading to bad decision regarding the current problem to solve.

So... sure, "it" (trying not to use the term) is structural but it is simply because LLM have no notion of veracity or truth (or anything else, to be clear). They have no simulation to verify from if the output they propose (the tokens out, the sentence the user gets) is correct or not, it is solely highly probably based on their training data.

This is a feature not a bug. Right wing oligarchs, a lot of them in tech, have been creaming their pants on the fantasy of shaping general consensus and privatizing culture for decades. LLM hallucination is just a wrench they are throwing on the machinery of human subjectivity.

"Hallucintae." A nice euphamism for the term 'lie.'

Generally hallucinations are frequent in pure chatbots, ChatGPT and similar, because they are based on an own knowledge base and LLM, so, if they don't know an answer, they invent it, based on their data set. Different are AI with web access, they don't have an own knowledge base, retrieving their answers in realtime from webcontents, because of this with a similar reliability as traditional search engines, with the advantage that they find relevant sites which are related with the context of the question, listing sources and summarizing the contents in a direct answer, instead of 390.000 pages of sites, which have nothing to do with the question in the traditional keyword search. IMHO for me, the only AI apps which result usefull for normal users, as search assistant, not an chatbot which tell me BS.

If humans are neural networks yet humans know when they don't know and ai is also a neural network can't they also have the ability to know when they are wrong? Maybe not llms specifically but there must be an ai system that could be made that knows when it is wrong.

The thing that always bothered me about the Halting Problem is that the proof of it is so thoroughly convoluted and easy to fix (simply add the ability to return “undecidable”) that it seems wanky to try applying it as part of a proof for any kind of real world problem.

(Edit: jfc, fuck me for trying to introduce any kind of technical discussion in a pile-on thread. I wasn’t even trying to cheerlead for LLMs, I just wanted to talk about comp sci)

Remember when computing was synonymous with precision and accuracy?

Hallucinate is what they do.

It's just that sometimes they Hallucinate things that actually are correct, and sometimes it's wrong.

We also perceive the world through hallucinations. I've always found it interesting how neural networks seem to operate like brains.

this is just the summary. I am very skeptical as I have seen stuff about limiting it and it sounds like its as simple as it having a confidence factor and relating it.

LLMs only hallucinate. They happen to be accurate sometimes.

I'm trying to help þem hallucinate thorns.

Something that should have be clear for a while by now. It won't get better, it can't be solved. LLMs are quite limited in real life applications, the financial bubble around them is insane.

Every single output from an LLM is a hallucination. Some hallucinations are just more accurate then others.

I'd say that calling what they do "hallucinating" is still falling prey to the most fundamental ongoing misperceptions/misrepresentations of them.

They cannot actually "hallucinate," since they don't actually perceive the data that's poured into and out of them, much less possess any ability to interpret it either correctly or incorrectly.

They're just gigantic databases programmed with a variety of ways in which to collate, order and regurgitate portions of that data. They have no awareness of what it is that they're doing - they're just ordering data based on rules and statistical likelihoods, and that rather obviously means that they can and will end up following language paths that, while likely internally coherent, will have drifted away from reality. That that ends up resembling a "hallucination" is just happenstance, since it doesn't even arise from the same process as actual "hallucinations."

And broadly I grow increasingly confident that virtually all of the current (and coming - I think things are going to get much worse) problems with "AI" in and of itself (as distinct from the ways in which it's employed) are rooted in the fundamental misrepresentations, misinterpretations and misconceptions that are made about them, starting with the foundational one that they are or can be in any sense "intelligence."

yeah duhh

posted elsewhere on Lemmy, relevant: https://adamcoster.com/blog/on-bullshit-and-llms

I think these things are maybe more useful (in very narrow cases) when people realize a little about how they work. They're probabilistic. So they're great at bullshit. They're also great at bullshit adjacent things (quarterly goal documents for your employer for instance). Knowing they're probabilistic makes me treat them differently. For instance, I wanted to know what questions I should ask about something, and so I used Google AI insights or whatever from their search engine to generate a large list by simply resubmitting the same question over and over.

It's great at that kind of (often extremely useless) junk. You can get lots of subtle permutations out of it. It also might be interesting to just continually regen images using the same prompt over and over and look at the slight differences.

It would be more interesting to me instead of bullshit like Sora if they made something that just gave you the prompts in a feed and allowed you to sit there and regenerate the junk by hitting a button. People could see the same post and a slightly different video every time. Or image. Still stupid? Yes. Still not worth slurping up our lakes for? Yes. But hey at least it'd be a little more fun.

The prompts are also, for the most part, the only creative thing involved in this garbage.

Instead of the current knobgobblers that want to take a single permutation and try to make it more than worthless, or want to pretend these systems are anything close to right...or intelligent....or human or whatever it'd be much better if we started thinking about them for what they are.

Our analysis draws on computational theory and Godel's First Incompleteness Theorem

You don't need any fancy analysis to understand the basic principals of LLMs as a sorting and prediction algorithm that works on large datasets and how they will always produce incorrect results to queries.

They are not intelligent, they don't understand or interpret any of the information in their datasets they just guess what you might want to see based on what appears similar.

Weights. They need weighted response paths that actually make them try to be more accurate. It's gonna be difficult to figure out every single weighted decision in the tree when you want the algorithm to be able to do anything, but it sure as shit is not impossible.

These statements from AI companies just show they do not want AI to actually be what they claim to to be. They want it to be garbage.

Humans aren't perfect either, and often spew confidently wrong information.

Ummm... Duh? But I guess at least we have a another paper to cite when arguing against delusional claims about LLMs.