98
A 25-Person Startup Built a Chip That Only Runs One AI Model. It's 73 Times Faster Than Nvidia. (dev.to)
submitted 8 months ago by to c/technology@beehaw.org
43 comments fedilink hide all child comments
 

Taalas HC1: 17,000 tokens/sec on Llama 3.1 8B vs Nvidia H200's 233 tokens/sec. 73x faster at one-tenth the power. Each chip runs ONE model, hardwired into the transistors.

you are viewing a single comment's thread
view the rest of the comments
[–] 21 points 8 months ago (2 children)

Each chip runs ONE model, hardwired into the transistors.

That's... that's an ASIC. That's literally just an ASIC... with all the tradeoffs and compromises that come with it.

  • permalink
  • fedilink
  • source
  • hideshow 2 child comments
    •
  • [–] 13 points 8 months ago

    Shh you'll pop the bubble if you start talking sensibly. It's not an ASIC—it's a specialized piece of hardware optimized to execute a model with unparalleled performance. Now buy my entire stock of them and all the supply for the next two years please.

    (Figuring out the compose combination for an emdash took longer than I'd like to admit lol)

  • permalink
  • fedilink
  • source
  • parent