I have avoided AI (LLMS) because it didn't seem nearly useful enough to warrant supporting these tech assholes. Is there anyway to use it locally on a normal device (16 Gb ram, 8 year old Geforce graphics, etc?). Can I just slow it down and get it to work locally?

you are viewing a single comment's thread
view the rest of the comments
[–] 11 points 1 week ago

Unsloth Studio is one of the easiest and most approachable methods of doing what you want. There are some smaller models that will run fine on the hardware you described, but they won’t be particularly “smart”. Still, they might work for what you need.

It’s important to bear in mind that as we move forward, these open weight models will just get smaller, better, and more efficient. Data centers are absolutely the wrong approach for this tech, and the only reason it exists that way now is due to capital interests trying to steal all the value. It won’t last as the technology matures, so go ahead and at least familiarize yourself with some ways to run local because that is the future.

https://unsloth.ai/

  • source