26
[AIT] what is your selfhosted local LLM stack? (lemmy.blahaj.zone)
submitted 2 weeks ago by to c/selfhosted@lemmy.world
50 comments fedilink hide all child comments
 

What tools and projects do you use for your local LLM chats and agents?

I'm trying to sift through the mess of LLM harness and tools to find how to really get the most out of local LLMs.

you are viewing a single comment's thread
view the rest of the comments
[–] 9 points 2 weeks ago (3 children)

@FauxLiving @YeahToast "I'm sure my driving to work will doom the world" same mentality.Also, it's not so much use of a model as the creation of the model. Training is expensive in terms of resources and energy. Using the model and normalizing AI usage leads to more AI usage and more AI normalization, which leads to new models being trained..

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 3 points 2 weeks ago (2 children)

    Put it into perspective:

    • Everyone doing their laundry uses more water than all data centers combined. Golf courses also use vastly more water.
    • Data centers use about 1-2% of global electricity.

    People driving cars pollutes more as does global shipping, regular old consumerism, etc. The environmental impact of data centers is negligible in the grand scheme of things.

    Now that AI is making open source software the most secure software ever written (the hard way, hehe), not using it to scan your stuff is now considered worst practices and basically idiotic negligence. No company that uses software or anything with firmware can afford to ignore it anymore.

    There's no way out of it. Accept the new AI reality and push for good regulations. Example: If we made it so that all AI data centers had to be powered via renewable energy on site that would allay most of the energy use/environmental concerns.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 6 points 2 weeks ago (1 child)

    I don't buy it. If Datacentres were the powersipping butterflies you suggest, why all the new fossil power plants everywhere? I would probably not have given a shit about the datacentres, if not for all the new fossil pollution that is part and parcel of the construction.

    Lies, damn lies, and statistics.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 4 points 2 weeks ago

    Here's the thing, datacenters could be powersipping butterflies if they built solar farms and ground source heat pumps when they build out the data centers. But the Epstein class are too cheap, greedy and selfish to do that without being forced to. They are accustomed to having the government, directly or indirectly, lower their costs and guarantee them profits.

  • permalink
  • fedilink
  • source
  • parent