I am looking at resintalling my OS (currently Mint) which I have been running for a few years.

Mint is great and I may end up going back; but was thinking of trying something new.

I want to be able to play with local models and agents; I have a pretty decent system.

  • AMD 3700X
  • 64 GB DDR4
  • AMD RX9070 16GB GPU
  • 2TB NVME SSD
  • A few TB of HDD's

What are some good options for easily running models and agents; I have been using ollama to run models; but I'm sure things have moved on since I begun playing with things a year ago.

Give me a few tips; I am going to have a bit of time on my hands over the next few months.

you are viewing a single comment's thread
view the rest of the comments
[–] [S] 2 points 4 days ago (2 children)

I was going to do a reinstall; it has been a few years of inplace upgrades and a lot of extra cruft that inevitably gets built up. I was asking mainly to see if there was a clear recommendation for which system is best for experimenting with local LLM's etc...Looks like there isn't any specific advantages to any system so I'll probably just stick with Mint, likely wait until Mint 27 is released before doing an upgrade.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 3 days ago (1 child)

    Ah, I was worried that there was some local model distro meta that I wasn't aware of. I have no knowledge of AMD but according to DeepSeek we seem to have somewhat comparable hardware for local LLM. I just noticed the 64GB RAM and figured you'd be on a whole different level, but apparently that setup is in the mid-high tier, spanning from 7-32B? What kind of models have you tried so far, and did any stand out? I've only been able to get a 13B Qwen3.5 model as most myself (which was way too slow to be worth it). From what I have picked up >24B is where you get a generally good model that goes beyond a niche purpose.

  • source
  • parent
  • hideshow 1 child comment
  • [–] [S] 2 points 3 days ago

    It has been a while, but last model I ran was gemma4:e4b 9.6GB. It ran very quickly; also have run deepseek-coder-v2:16b 8.9GB, again very fast.....

    I am looking to get back into it once I have a bit of time on my hands; I'll look into the state of the art models again and see what I can run.

  • source
  • parent