Ollama for API, which you can integrate into Open WebUI. You can also integrate image generation with ComfyUI I believe.

It's less of a hassle to use Docker for Open WebUI, but ollama works as a regular CLI tool.

!localllama@sh.itjust.works

I've discovered jan.ai which is far faster than GPT4All, and visually a little nicer.

EDIT: After using it for an hour or so, it seems to crash all the time, I keep on having to reset it, and currently am facing it freezing for no reason.

Maybe LocalAI? It doesn't do python code execution, but pretty much all of the rest.

The main limitation is the VRAM, but I doubt any model is going to be particularly fast.

I think phi3:mini on ollama might be an okish fit for python, since it's a small model, but was trained on python codebases.

You can tell Open Interpreter to run commands based on you human-language input. If you want local only LLM, you can pair it with Ollama. It works for "interactive" use where you're asked for confirmation before a command is run.

I set this up in a VM because I wanted a full automatic coding "agent" which can run commands without my intervention and I did not want it to blow up main system. It did not really work though because as far as I know Open Interpreter does not have a way to "pipe" a command's output back into the LLM so that it could create feedback with linters and stuff.

Another issue was that Starcoder2, which is the only LLM trained on permissive licensed code I could find, only has a 15B "human-like" model. The smaller models only speak code so I don't know how that would work for agentic usage and the 15B is really slow running on DDR4 CPU. I think agents are cool though so I would like to try Aider which is a supposedly good open source agent and unlike Open Interpreter is not abandonware.

Thanks for coming to my blabering talk, hope this might be useful for someone.

AUTOMATIC1111?

https://github.com/AUTOMATIC1111/stable-diffusion-webui

You should try https://cherry-ai.com/ .. It's the most advanced client out there. I personally use Ollama for running the models and Mistral API for advnaced tasks.