62
In search for a new self-hosted LLM (lemmy.ml)
submitted 7 months ago by to c/selfhosted@lemmy.world
30 comments fedilink hide all child comments
 

Hey :) For a while now I use gpt-oss-20b on my home lab for lightweight coding tasks and some automation. I'm not so up to date with the current self-hosted LLMs and since the model I'm using was released at the beginning of August 2025 (From an LLM development perspective, it feels like an eternity to me) I just wanted to use the collective wisdom of lemmy to maybe replace my model with something better out there.

Edit:

Specs:

GPU: RTX 3060 (12GB vRAM)

RAM: 64 GB

gpt-oss-20b does not fit into the vRAM completely but it partially offloaded and is reasonably fast (enough for me)

you are viewing a single comment's thread
view the rest of the comments
[–] 32 points 7 months ago (2 children)

I find Qwen3.5 is the best at toolcalling and agent use, otherwise Gemma4 is a very solid all-rounder and it should be the first you try. Tbh gpt-oss is still good to this day, are you running into any problems w it?

  • permalink
  • fedilink
  • source
  • hideshow 2 child comments
    •
  • [–] [S] 7 points 7 months ago (1 child)

    No problems per se. I just thought that I had not checked for an update for a longer time.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 2 points 7 months ago

    You're probably aware, but updating the model periodically is probably a good idea just because things do change overtime.

    A model from two years ago was trained on data from at least two years ago. Meaning any technology, code, world event changes wouldn't be reflected in the model.

  • permalink
  • fedilink
  • source
  • parent