[–] 3 points 5 months ago* (last edited 5 months ago) (2 children)

I started working toward self hosting LLM for my small company using ollama and opencode as agent But I realized a good model like GLM 5 requures 250GB of RAM and 24GB vram with a 4080?? I dont know, this is what the LLM told me itself.

I ended up using qwen-code2.7-7b-16k.

Currently the best thing I have is my laptop, 16GB ram, i7 9750H gtx1650

How do you guys selfhost? What models do you use that are actually good?

  • source
  • view more: ‹ prev next ›