dieICEdie comments on A 25-Person Startup Built a Chip That Only Runs One AI Model. It's 73 Times Faster Than Nvidia.

A 25-Person Startup Built a Chip That Only Runs One AI Model. It's 73 Times Faster Than Nvidia. (dev.to)

submitted 8 months ago by mothasa@x69.org to c/technology@beehaw.org

43 comments fedilink hide all child comments

Taalas HC1: 17,000 tokens/sec on Llama 3.1 8B vs Nvidia H200's 233 tokens/sec. 73x faster at one-tenth the power. Each chip runs ONE model, hardwired into the transistors.

[–] dieICEdie@lemmy.org 1 point 8 months ago (6 children)

But if they could make it so the chip is the only thing that is obsolete, That could be recycled pretty easily, or resold.

permalink

fedilink

source

parent

hideshow 6 child comments

[–] FurryMemesAccount@lemmy.blahaj.zone 1 point 8 months ago (5 children)

Then it would stop being 73 times faster than NVIDIA.

permalink

fedilink

source

parent

hideshow 5 child comments

[–] dieICEdie@lemmy.org 1 point 8 months ago (4 children)

That doesn’t make sense.

permalink

fedilink

source

parent

hideshow 4 child comments

[–] FurryMemesAccount@lemmy.blahaj.zone 1 point 8 months ago (3 children)

If you add levels of indirection, extra transistors and such, it would be surprising to manage to maintain the same level of performance, especially since this design seems to rely on hardwiring to achieve its speed...

permalink

fedilink

source

parent

hideshow 3 child comments