287
Character.ai has initiated voluntary Chapter 11 bankruptcy (blog.character.ai)
submitted 3 weeks ago by to c/technology@lemmy.world
44 comments fedilink hide all child comments
 
you are viewing a single comment's thread
view the rest of the comments
[–] 16 points 3 weeks ago* (5 children)

The environmental impact of AI would be a tenth of what it is now if people just ran whatever models they want on their own local machines. We're mortgaging the next fifty years worth of compute power and meanwhile mfs who talk to virtual girlfriends on their $2,000 gaming PC have trillions of FLOPS of compute power sitting right next to them.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 5 child comments
    •
  • [–] 12 points 3 weeks ago (3 children)

    Why is that? The electricity burned is about the same, no? Of course, the impact is more distributed.

    I think the difference would be that local machines aren't subsidized, so you'd see people get a lot more skittish after that first electric bill. Data centers are like railroad tracks from the railroad boom (if they'd made the rails out of licorice twizzlers.)

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 8 points 3 weeks ago* (last edited 3 weeks ago)

    With local you're limited in performance by the hardware you can buy, which is a huge up-front that most people would balk at. Hosted will give you as much performance as you could ever want billed by usage with several profit vectors built-in, but the customer sees smaller dollar number and better apparent performance and thinks they're getting a deal. Car dealerships do the same thing with selling the monthly payment and not the actual sticker price.

    The environmental difference then could be significant. Users taking the hosted route can choose to use much more resource-intensive models than a local PC could. They might see a $300 AI bill as a little pricey when it is usually $100, but figure that's cheaper than buying a PC that could produce the same result, just at maybe half speed. This would result in a much higher energy cost from the hosted solution compared to the PC.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 4 points 3 weeks ago* (last edited 3 weeks ago)

    Electricity burned is the same. But so is the electricity burned while gaming, your system is limited by your GPU in both instances. If you have an RTX 5090 then you can throw almost 600 watts continuous at both which could use like 14kW/day, but most people will have more modest hardware, and therefore the energy usage of running a local model wouldn't be all that much either - I don't think many people limit their gaming because of how much the electricity would cost.

    But when you have a datacenter with hundreds of servers capable of maxing out the residential house mains of half a dozen houses each (single NVIDIA NVL72 AI server has 72 GPUs and uses 120-150kW at full load) they can throw way, way more compute at you than you ever could manage locally.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 3 points 3 weeks ago

    Why is that? The electricity burned is about the same, no? Of course, the impact is more distributed.

    Forget about everything related to the prompt itself being processed for the moment. What other information do you think they are grabbing and processing along with the prompt? Maybe some data about you and your interactions for resale? Account info? Piecing together details about you perhaps, for some of that highly-targeted-advertising-data companies love to sell?

    Even the comment you are replying to, IMO, is pretty conservative about the actual energy savings.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 4 points 3 weeks ago* (last edited 3 weeks ago)

    That doesn't check out when considering model size. Unless that's what you're trying to say that self-hosted models force people to use smaller models. Just from an operations perspective a data center had economies of scale and the ability to batch up multiple independent user's requests which should reduce the per token kWh along with the fact that your utilization will be higher vs somebody's GPU at home would be idling.

    That said I do believe there are negative environmental impacts from AI like always using giant frontier models for everything instead of smaller fine tuned models.

    EDIT: I see a bunch of replies on the other comment, which all raise valid points, but that's definitely not the way most readers are going to interpret the original post and moves the goal posts.

  • permalink
  • fedilink
  • source
  • parent