you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 5 months ago*

For self hosting it essentially needs to fit in VRAM + RAM but it'll take a lot of CPU for the part in RAM

Deepseek probably uses those big fancy H cards and not one but several together to increase VRAM.

  • source
  • parent