As the title says, i started by selfhosting OpenWebUI including Ollama on my RIG. I have been pretty happy but the more i dig into this stuff, more i understand that i am doing it wrong and i definitely need to switch to llama.cpp / ik_llama.cpp.

But i have a few questions...

  1. I want a web based LLM chat GUI, because that's my 80% usage for AI. If i go with llama.cpp, do i need to ditch OpenWebUI as well? Is there a better UI? Do i need an UI?

  2. i am currently hosting it all with a docker compose file. Is this still doable if i switch? I can go bare-metal (Gentoo server, good skills on my side) but it's the maintenance part, a "podman compose pull" is just easier... or i am lazy.

  3. the server is headless and always accessed remotely via web or ssh, just to be clear.

My hardware is a NVIDIA RTX A4000 16GB VRAM on a I7-8700@3200Ghz with 64GB system RAM (shared with far too many services).

you are viewing a single comment's thread
view the rest of the comments
[–] [S] 2 points 4 months ago

I gave it a spin today and pretty happy so far. Took a bit to run, llama.cpp got installed broken and had to reinstall it ...

But it pretty much replaced my entire openwebui+ollama stack with something seemingly much more capable and open.

  • source
  • parent