89
I made my LLM stop bullshitting. Nothing leaves your machine. (codeberg.org)
submitted 7 months ago by to c/privacy@lemmy.ml
46 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[+] [S] 4 points 7 months ago* (last edited 6 months ago) (1 child)
[deleted]
  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 2 points 7 months ago

    The problem with CAG is not just that it hogs memory, but to keep it fresh you have to keep re-indexing. If the corpus is large and dynamic, it can easily fall out of date and, at runtime, blow out the context window.

    GraphRAG has some promise. NVidia has a playbook for converting text into a knowledge graph: https://build.nvidia.com/spark/txt2kg

    It'll probably have the same issues with reindexing, but that will be a common problem, until someone comes up with better incremental training/indexing.

  • permalink
  • fedilink
  • source
  • parent