268
If open weight models are the future, U.S. AI companies are going to have a hard time (www.fastcompany.com)
submitted 3 months ago by to c/technology@lemmy.world
86 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 3 months ago* (1 child)

It's not necessarily that any of the open models are cheaper to train (for similar model size), it's more that China has deeper pockets. And MoE inference is cheaper than dense models I believe

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 2 points 3 months ago

    If China manages to scale their hardware production, this could finally be "their day in the sun" where they clearly surpass the West in the way the West has outshone them for 100 years. MoE has its place, but a MoDE where each E is itself a dense model would be more powerful still, mostly you need the silicon gates and power to drive them.

    Three Gorges makes hydro-power, right? 22MW -> 100 TWh per year, just from that one structure, I bet that will run at least one AGI... https://www.fredgao.com/p/deepseeks-liang-wenfeng-breaks-his

  • permalink
  • fedilink
  • source
  • parent