2
An idea for saving cost. Local AI model use. (lemmy.world)
submitted 1 month ago by to c/perchance@lemmy.world
2 comments fedilink hide all child comments
 

So perhaps I could be fooling myself, or maybe this is a bad idea...

But why not allow us to use local models as the helper? If we just added a pipeline to a local model as the brain of the helper, and kept all the resources that the old helper uses as a reference... That might alleviate a lot of stress and some cost off the function of the AI itself, which could save some money would it not?

I'm not super savvy on all the technical background required but as a concept I think this might at least somewhat help save on the COST side of things.

Thoughts?

you are viewing a single comment's thread
view the rest of the comments
[–] [S] 1 point 1 month ago

That's fair, but a big thing I'm also thinking of is server load. I know many people run several page tabs at once (A person I was talking to was running 10 helpers at once on the same project... Don't ask. He was doing a lot of stuff at once.) and that rubbed me a bit bad since perchance is already struggling with high server usage.

Being able to use local models (or the like) would really help remove a lot of server load. (As well as cost... The cost was more the after-thought during the conception of the idea, but I figured people always think money-first.)

  • permalink
  • fedilink
  • source
  • parent