If I can pick my own API (including local) and sampling parameters
You can do this now:
- selfhost ollama.
- selfhost open-webui and point it to ollama
- enable local models in about:config
- select "local" instead of ChatGPT or w/e.
Hardest part is hosting open-webui because AFAIK it only ships as a docker image.
Edit: s/openai/open-webui