60
AI Is A Money Trap (www.wheresyoured.at)
submitted 1 year ago by to c/technology@beehaw.org
25 comments fedilink hide all child comments
 

As always with Zitron, grab a beverage before settling in.

you are viewing a single comment's thread
view the rest of the comments
[–] [S] 2 points 1 year ago (7 children)

I don't think it's going to come down to these absurd datacentres. We're only a few years off from platform-agnostic local inference at mass-market prices. Could I get a 5090? Yes. Legally? No.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 7 child comments
    •
  • [–] 5 points 1 year ago (1 child)

    I have to think that most people won't want to do local training.

    It's like Gentoo Linux. Yeah, you can compile everything with the exact optimal set of options for your kit, but at huge inefficiency when most use cases might be mostly served by two or three pre built options.

    If you're just running pre-made models, plenty of them will run on a 6900XT or whatever.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] [S] 3 points 1 year ago

    I don't expect anyone other than ... I don't even know what the current term is ... geeks? batshit billionaires? to be doing training.

    I'm very much of the belief that our next big leap in LLMs is local processing. Once my interactions stay on my device, I'll jump in.

  • permalink
  • fedilink
  • source
  • parent
  • [+] 3 points 1 year ago (4 children)
    [deleted]
  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] [S] 3 points 1 year ago (3 children)

    There are already large local models. It's a question of having the hardware, which has historically gotten more powerful with each generation. I don't think it's going to be phones for quite some time, but on desktop, absolutely.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [+] 2 points 1 year ago (2 children)
    [deleted]
  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] [S] 3 points 1 year ago

    For the security tradeoff of sensitive data not heading to the cloud for processing? Not all businesses, but many would definitely see value in it. We're also discussing this as though the options are binary ... models could also be hosted on company servers that employees VPN into.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 1 point 1 year ago*

    Keywords: NPU, unified RAM

    Apple is doing it, AMD is doing it, phones are doing it.

    GPUs with dedicated VRAM are an inefficient way of doing inference. They've been great for research purposes, into what type of NPU may be the best one, but that's been answered already for LLMs. Current step is, achieving mass production.

    5 years sounds realistic, unless WW3.

  • permalink
  • fedilink
  • source
  • parent