75
What budget friendly GPU for local AI workloads should I aim for? (lemmy.dbzer0.com)
submitted 1 year ago by to c/selfhosted@lemmy.world
32 comments fedilink hide all child comments
 

Hiya,

Recently upgraded my server to an i5-12400 CPU, and have neen wanting to push my server a bit. Been looking to host my own LLM tasks and workloads, such as building pipelines to scan open-source projects for vulnerabilities and insecure code, to mention one of the things I want to start doing. Inspiration for this started after reading the recent scannings of the Curl project.

Sidenote: I have no intention of swamping devs with AI bugreports, i will simply want to scan projects that i personally use to be aware of its current state and future changes, before i blindly update apps i host.

What budget friendly GPU should i be looking for? Afaik VRAM is quite important, higher the better. What other features do i need to be on the look out for?

you are viewing a single comment's thread
view the rest of the comments
[–] 10 points 1 year ago (3 children)

It's all VRAM, that's the bottleneck for even the best GPUs. AMD support is spotty so you should stay in Nvidia's claws unless you know what you're doing. Figure out what kind of money you're willing to part with, and then get whatever Nvidia GPU gets you the most VRAM.

  • permalink
  • fedilink
  • source
  • hideshow 3 child comments
    •