268
If open weight models are the future, U.S. AI companies are going to have a hard time (www.fastcompany.com)
submitted 3 months ago by to c/technology@lemmy.world
86 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 3 points 3 months ago (2 children)

The other problem is bandwidth. Even if you can run Kimi K3 locally, AI usage patterns tend to be very bursty. Having the scalability of cloud compute to run multiple agents and sessions at once makes it much more usable.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 4 points 3 months ago (1 child)

    My current workflow is passing a human written spec to an agent to implement with strict coding guidelines, architectural decisions, etc. The agent isn't making any decisions about the abstractions to use, it's just creating the objects and test suites. So I don't mind the slower bandwidth because I'm running the heavy agentic lifting over night with no need for human supervision.

    But I fully appreciate that my workflow isn't the norm. In fact my workflow it's the exact opposite the AI grifters like Sam Altman are selling because it still involves a human with knowledge of the systems making different decisions.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 3 points 3 months ago*

    If you're running stuff overnight on a homelab, you can get away with using very large models as agents in CPU RAM.

    Another option is a hybrid approach: farm "dumb" privacy-insensitive calls to an API you can subscribe to for like $40 a year, and let your local LLM do the orchestrating. This is what I do: I run MiMo locally, but also farm calls out to the exact same LLM in the cloud when appropriate... And I don't mind one bit. I'm happy to tip them for open sourcing it in the first place.

  • permalink
  • fedilink
  • source
  • parent