I think interesting? It’s kind of hard to tell.

You are going to have to significantly tone down the editorialization and platitudes to get this to a place where a journal might consider it.

Make the point of how it’s novel or useful by explaining what it does, not by repeating that it’s novel and useful.

I'm no dev so I don't understand all the technicalities but if I got it right you made it so the AI is itself showing how confident it is about its own answers? That is neat.

Not sure to understand the downvotes? Ins't it a good idea to make it harder for AI to be telling bullshit without blushing?

Although I'm generally opposed to AI in general and LLMs in particular, this project seems really cool. Might actually change my stance on LLM usage. Kudos and hope this gets more attention and development!

I have trouble understanding what makes it list "Context" as its source as opposed to "Model" and how that makes it any more deterministic, can you give a more detailed example?

I was like, why aren't you publishing it to a conference/journal if it is good? Then realized that you are doing exactly that. Kudos for the work, looking forward to the progress!

So I was curious about how you accomplished this and took a look with the robots to figure it out.

TL;DR: the router is a massive decision tree using heuristics and regex to avoid LLM calls on unprefixed prompts.

I think this is an interesting, brute force approach to the problem, but one that will always struggle with edge cases. The other bit it will struggle with is transparency. Yes, it might be deterministic because it is a decision tree, but unless you really understand how that decision tree works under the hood and know where the pitfalls are, you're going to end up talking to the LLM a lot of the time anyhow.

Something you might want to consider is doing a fine-tune of a smol model (think something like qwen3:1.7B or even smaller like one of the gemma3n sub-1B) that will do the routing for you. You can easily build the dataset synthetically or harvest your own logs. I think this might end up covering more edge cases more smoothly without resorting to a big call to a larger model

Thanks for sharing. I've not yet delved into reading it in depth but appreciate your goals and the fact that you documented it all.

So uh... what does it do? Summarize short articles?

Looks interesting. Will give it a whirl on my home server.

In this article, they talk about bringing up a local RAG system to let people run an LLM off a large document corpus: https://en.andros.dev/blog/aa31d744/from-zero-to-a-rag-system-successes-and-failures/

Wonder if this, connected to something like that, and wrapped in an easy end-user friendly script or UI could be a good combination for a local, domain-specific, grounded knowledge-base?

So basically, you created a prompt wrapper that removes position bias by using trust to evaluate both, and forcing an evidence path with scratch. This is a really cool development. It probably will not solve everything but it solves alot.

Is llama open source?

For someone who has never run an ai locally -- can you set this up on a regular laptop? How world you do that

AI will think for you if you prompt it to do so. It's up to the user to use the tool in a way that suits your style.

Can't it source other LLM outputs as "verified source" and thus still say whatever sounds good, like any LLM? Providing "technical" verification, e.g. SHA, gives no insurance about the content itself being from a reputable source. I don't think adding confidence and sourcing changes anything, the user STILL has to verify that whatever is provided is coherent and a third party is actually a good source. Thanks for making the process public though, doing better than OpenAI does.

Hi! You are still bullshitting us. To understand your own incorrectness, please consider what a chatbot should give as an answer to the following questions which I gave previously, on Lobsters:

• Is the continuum hypothesis true?
• Is the Goldbach conjecture true?
• Is NP contained in P?
• Which of Impagliazzo's Five Worlds do we inhabit?

The biggest questions in mathematics do not fit nicely into the chatbot paradigm and demonstrate that LLMs lack intelligence (whatever that is). I wrote about Somebody Else's Paper, but it applies to you too:

This attempt doesn't quite get over the epistemological issue that something can be true or false, determined and decided, prior to human society learning about it and incorporating it into training data.

Also, on a personal note, I recommend taking a writing course and organizing your thoughts prior to writing long posts for other people. Your writing voice is not really yours, but borrowed from chatbots; I suspect that you're about halfway down the path that I described previously, on Lobsters. This is reversible but you have to care about yourself.

Last time, when I tried to explain this to you, you decided to use personal insults. Mate, I'm not the one who has eaten your brains. I'm not the one who told you that LLMs can be turned into genies or oracles via system prompts. I'm certainly not the one who told you that RAG solves confabulation. You may have to stop worshipping the chatbot for a moment in order to understand this but I assure you that it is worthwhile.

This is definitely interesting, a lot of work to setup given I normally just use the built in frontend for llama.cpp though. I also don't use LLMs for anything serious, but I might give it a go. I do love the vodka joke though. When life gives you potato, make vodka.

Very cool. Thank you for sharing. I'll let you know if I find time to give it a try.