It matters a bit but not as much as you'd think, it also depends on what model and what runner you're using I think. There's a lot of optimizing that can be done, but in practice, you're not going to notice huge difference in speed. Again, remember that these are not speed-demon cards to begin with, the main feature is the large VRAM capacity that allows them to run very large, powerful models without the speed penalty of dropping to system RAM, and that, at least where I understand it, is where the PCIe starts to matter a lot more. That said, maybe I'm wrong. I'm not an expert at this stuff and my experience is limited to what little hardware in what limited configurations I have available personally. Best of luck in your adventures, anything we can do to democratize machine learning technology to more people is worth pursuing I think.
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments