you are viewing a single comment's thread
view the rest of the comments
[–] 115 points 3 months ago* (last edited 3 months ago) (7 children)

Basically Apple will be building the perfect computers to run local LLMs.

  • source
  • hideshow 7 child comments
  • [–] 90 points 3 months ago (2 children)

    One can only hope that it totally breaks the AI/LLM at industrial scale, so businesses can run their own AI systems with their own data sets.

    No more of this fucking datacanter horseshit.

  • source
  • parent
  • hideshow 2 child comments
  • [+] 3 points 3 months ago* (last edited 1 month ago) (3 children)
  • [–] 3 points 3 months ago

    It’s likely wrong reporting, the m5 ultra gets 614 GB/s today. The m3 pro gets over 800. The rumored m5 pro is 1.2TB/s. Likely this will be 2.4 TB/s, but half the reason for high bandwidth in nvidia chips is somewhat offset in the unified scheme. Mlx needs a lot less copy data around when the gpu can just read it directly.

  • source
  • parent
  • [–] 2 points 3 months ago*

    Something got reported on incorrectly because the M5 Max's have 614 GB/s today, and the Ultra M4 machine's (not laptops) are 819 and that's 3 generations behind a M7 if they make an M7 ultra machine.

    $ for $ you'll get more video ram than paying for a 5090, but it won't be as fast and can't train well.

    Before the ram price decable, you could get a 192gb M4 Ultra for ~10k CAD.

  • source
  • parent
  • [–] 1 point 3 months ago

    Only if they can figure out how to emulate CUDA. Or maybe more containers will start providing alternatives if the machines are popular enough?

    There’s still a lot of image and OCR workflows I have to chug through the CPU and it takes forever.

  • source
  • parent
  • load more comments (-1 replies)