you are viewing a single comment's thread
view the rest of the comments
[–] 3 points 11 months ago* (last edited 11 months ago)

Bloefz has a great setup. Used Mi50s are cheap.

An RTX 3090 + a cheap HEDT/Server CPU is another popular homelab config. Newer models run reasonably quickly on them, with the attention/dense layers on the GPU and sparse parts on the CPU.

  • source
  • parent