Dell is now shifting it focus this year away from being ‘all about the AI PC.’

you are viewing a single comment's thread
view the rest of the comments
[–] 42 points 10 months ago (23 children)

WTF even is an "AI PC"? I saw an ad for some AI laptop. To my knowledge, nobody is running LLMs on their personal hardware, so do these computers have like...a web browser?

  • source
  • hideshow 23 child comments
  • [–] 13 points 10 months ago* (last edited 10 months ago) (1 child)

    You can run it on your laptop, I've tried it before (PS e.g. https://www.nomic.ai/gpt4all, it's fun to dig through your documents, but it wasn't as useful as I thought it would be in collecting ideas), what is truly hard is to train. But yeah, what is an AI PC? Is it like a gaming rig with lotsa RAM and GPU(s)?

  • source
  • parent
  • hideshow 1 child comment
  • [–] 10 points 10 months ago (5 children)

    Computers now come with a NPU (Neural Process Unit) to do that job... So yeah.

  • source
  • parent
  • hideshow 5 child comments
  • [–] 9 points 10 months ago (4 children)

    What kind of consumer-facing software runs on that NPU?

  • source
  • parent
  • hideshow 4 child comments
  • [–] 9 points 10 months ago

    For what it’s worth an NPU is why your phone could tell you that photo is of a cat years before LLMs were the hot new thing. They were originally marketed as accelerators for machine learning applications before everybody started calling that AI.

  • source
  • parent
  • [–] 19 points 10 months ago (5 children)

    To my knowledge, nobody is running LLMs on their personal hardware

    They absolutely are.

  • source
  • parent
  • hideshow 5 child comments
  • [–] 14 points 10 months ago (4 children)

    Statistically relevant portion?

    You know they were hyperbolic.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 1 point 10 months ago (3 children)

    "To my knowledge" really doesn't feel like hyperbole at all, IMO

  • source
  • parent
  • hideshow 3 child comments
  • [–] 5 points 10 months ago (2 children)

    Do 5% of people you know use local LLMs?

    If so, you don't know a lot of people, or you are heavy into the lLLM scene.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 3 points 10 months ago (1 child)

    Do 5% of people you know watch hockey regularly? If not, I guess it must not be a real sport, and that definitely has absolutely nothing to do with your own bubble

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 10 months ago

    Surprisingly a lot of people around me watch hockey.

    And I also hear about a lot of people watching it.

    But even though I'm very much into IT, I know very few people who selfhost. Despite that being a small community, local genAI seems even smaller that that.

  • source
  • parent
  • [–] 8 points 10 months ago

    Running an LLM locally is entirely possible with fairly decent modern hardware. You just won't be running the largest versions of the models. You're going to run ones intended for local use, almost certainly Quantized versions. Those usually are intended to cover 90% of use cases. Most people aren't really doing super complicated shit with these advanced models. They're asking it the same questions they typed into Google before, just using phrasing they used 20+ years ago with Ask Jeeves.

  • source
  • parent
  • [–] 11 points 10 months ago (4 children)

    Lots of people are. Typically it means they have an NPU.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 5 points 10 months ago (3 children)

    I'm talking about an ad I saw on broadcast television during a football game. I don't think the broad market of people are downloading models from huggingface or whatever.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 1 point 10 months ago (2 children)

    The ad you saw said no one was running local AI?

  • source
  • parent
  • hideshow 2 child comments
  • [–] 5 points 10 months ago (1 child)

    The ad was people doing generic AI stuff. I think it was even showing Copilot.

    Either way, the marketing for AI is far to nebulous for it to matter. Just looking for the ad, I found plenty (like this one) that explicitly mention "on-device AI," but show people just searching for shit or doing nebulous office work. This ad even shows generating images in MS Paint which offloads the AI shit to the cloud.

  • source
  • parent
  • hideshow 1 child comment