I am looking at resintalling my OS (currently Mint) which I have been running for a few years.

Mint is great and I may end up going back; but was thinking of trying something new.

I want to be able to play with local models and agents; I have a pretty decent system.

  • AMD 3700X
  • 64 GB DDR4
  • AMD RX9070 16GB GPU
  • 2TB NVME SSD
  • A few TB of HDD's

What are some good options for easily running models and agents; I have been using ollama to run models; but I'm sure things have moved on since I begun playing with things a year ago.

Give me a few tips; I am going to have a bit of time on my hands over the next few months.

you are viewing a single comment's thread
view the rest of the comments
[–] 3 points 5 days ago* (last edited 5 days ago) (2 children)

No suggestions, but keep us updated on your experience with ROCm. I'd love to migrate over to AMD GPUs for local model training, but the architecture just doesn't seem to have the toolkit support I need to feel secure about such a decision.

  • source
  • hideshow 2 child comments
  • [–] [S] 5 points 5 days ago (1 child)

    I have been running models on it for a year....seems to be getting better and better; I never ran on Nvidia so I can't compare.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 3 points 5 days ago

    I'm pretty heavily invested into training image and video generation models, so my use cases might be more particular. I recently did a huge deepdive and decided I should just stick to CUDA for the time being, given the specific toolkits and software I leverage in my process.

  • source
  • parent