3
LLM with Web Search functionality (lemmy.ml)
submitted 4 months ago by to c/selfhost@lemmy.ml
33 comments fedilink hide all child comments
 

Conducting deep web searches and gathering sources is one of the main things I've been using LLMs for. How far away are we from being able to self-host something like Claude's web search capabilities? Or even just a service where I'd pay with my money instead of my data?

you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 4 months ago* (1 child)

To avoid context switching on the GPU. OpenWebUi for example uses it for memory and title generation.

Those are not performance critical and background tasks, so instead of slowing down qwen, we just outsource this stuff to the NPU.

Edit: see here for more details

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •