What helped me in my project was a RAG implementation. So instead of trying to have all the information in the prompt, which becomes inconsistent if it becomes too big (especially when working on local llm), I can have quite a big knowledge base in a RAG. Then when being promoted it's a two step process, first search the RAG for the best match in the knowledge base, then Feed that to the LLM to generate the answers. You could also store noteworthy stuff in the rag on the fly.
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
replies: