What tools and projects do you use for your local LLM chats and agents?

I'm trying to sift through the mess of LLM harness and tools to find how to really get the most out of local LLMs.

you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 2 weeks ago* (last edited 2 weeks ago)

front: open-webui skill servers: dockling - docs parsing for llm, searxng - searchengine for llm, open-terminal - linux terminal for llm (inside container) base: llama-swap Inside llama swap: qwen3-14b-q5, translator-nemo-12b, tower-plus-9b, translator-gemma4-26b, qwen3.6-35b-a3b llama stack: ROCm

Hardware: RX9070XT for llama-swap .

UPD: if you want easy way and run all in oneshot use https://github.com/Niko1221/Strata

  • source