I'm new to local llms, i'm running Debian on this machine. Where should I start and what model(s) could I get testing on it?

you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 5 days ago (1 child)

Basically equivalent to a GTX 1660. 192 GB/s is relatively slow, memory bandwidth being (usually) the bottleneck for LLMs.

  • source
  • parent
  • hideshow 1 child comment