I have avoided AI (LLMS) because it didn't seem nearly useful enough to warrant supporting these tech assholes. Is there anyway to use it locally on a normal device (16 Gb ram, 8 year old Geforce graphics, etc?). Can I just slow it down and get it to work locally?

you are viewing a single comment's thread
view the rest of the comments
[–] 6 points 6 days ago* (last edited 6 days ago)

I donate my compute to https://aihorde.net/ once in a while. got to play around with some cool AI bullshit there. Its fun to play around with.

Its all volunteer run. I got my steam deck pushing small updates to peoples like dnd rollplay kinda stuff. koboldcpp makes it easy to set up and you can use hugging face models or your own.

Theres some really excellent open models out there if you care to look! Most are hyper specialized better than say GPT and such.

  • source