Deleted (aussie.zone)
๐Ÿ‘ 1
you are viewing a single comment's thread
view the rest of the comments
[โ€“] 3 points 5 months ago* (last edited 5 months ago) (2 children)

I started working toward self hosting LLM for my small company using ollama and opencode as agent But I realized a good model like GLM 5 requures 250GB of RAM and 24GB vram with a 4080?? I dont know, this is what the LLM told me itself.

I ended up using qwen-code2.7-7b-16k.

Currently the best thing I have is my laptop, 16GB ram, i7 9750H gtx1650

How do you guys selfhost? What models do you use that are actually good?

  • source
  • hideshow 2 child comments
  • [+] [S] 3 points 5 months ago* (last edited 5 months ago) (2 children)
  • [โ€“] 1 point 5 months ago (1 child)

    Qwen 3.5 24B is way too large for my specs. I'm barely running qwen2.5 7B

  • source
  • parent
  • hideshow 1 child comment
  • load more comments (-1 replies)
  • load more comments (-1 replies)