I am looking at resintalling my OS (currently Mint) which I have been running for a few years.

Mint is great and I may end up going back; but was thinking of trying something new.

I want to be able to play with local models and agents; I have a pretty decent system.

  • AMD 3700X
  • 64 GB DDR4
  • AMD RX9070 16GB GPU
  • 2TB NVME SSD
  • A few TB of HDD's

What are some good options for easily running models and agents; I have been using ollama to run models; but I'm sure things have moved on since I begun playing with things a year ago.

Give me a few tips; I am going to have a bit of time on my hands over the next few months.

you are viewing a single comment's thread
view the rest of the comments
[–] 6 points 5 days ago* (last edited 5 days ago)

I used a mix of Ubuntu, Fedora, and Windows for a decade or so but recently switched over to CachyOS and it's pretty nice, installing / updating graphics drivers is easy as and having access to the AUR is pretty neat.

Switching to a new distro would also be a good time to explore better alternatives to ollama too, plain llama.cpp is decent (got it running on my laptop rtx 3060) and Strata is quite neat for supported MoE models - runs a bit faster than llama.cpp on my oculink 9060 xt + 64gb ddr4.

  • source