you are viewing a single comment's thread
view the rest of the comments
[–] 36 points 1 year ago (1 child)

39 GB is very small, DeepSeek R1 without quantization at full context size needs almost a full TB of RAM/VRAM.

The large models are absolutely massive and you will still find some crazy homelabber that does it at home.

  • source
  • parent
  • hideshow 1 child comment