Jesus that is bonkers. 480GB on one GPU? I get why (model size and context headroom)… but that’s gonna be closing in on $100k (MSRP of an NVDA B300 - iirc their current top-end - is $50-60k, and that only (hah) brings 288GB).

Tangentially: I really wish the big US LLM players would stop fucking brute forcing their development by throwing silicon at everything. It seems embedded/austere/efficient coding techniques and algorithms are just being straight up discarded and forgotten. Tbh, this is a pattern I’ve observed as a slow creep since the advent of AWS and The Cloud a couple decades ago - the “AI” bubble is just accelerating the trend. The narrative (irritatingly parroted by my manager) is always “make it work, then make it efficient later” - but it’s always put in the tech debt bucket, so “later” is actually “never”. China is absolutely running way ahead of the US these days in terms of nurturing incisive and efficient engineering culture - it’s not even close. The fact that it’s essentially borne out of ITAR restrictions on the most advanced GPU tech is incidental; really, in a pretty significant way, the US kinda unintentionally did them a favor by constraining their hardware tier, because it forced them to write better code to make up for it. And they have been writing better code for a while now, especially in the LLM domain.

They could have made 30 modern GPUs for that amount of ram. It's not about the money.

I wonder if those will support Vulkan for their eventual use in gaming when they become outdated

That was my first question as well. I dont remember seeing anything on the C series

There is an actual hardware reason for a lot of this unfortunately. Some apis are emulated in the drivers on cpu or gpu shaders instead of being implemented properly in dedicated hardware paths. This is especially true for dx9 and dx10, partially true for some uncommon dx11 and dx12 functions.

Iirc battlemage is the first Intel GPU line that has proper hardware implementations for the entire API, or at least complete enough that it doesn't massively affect performance.

Fascinating read/watch if you want to learn more

https://devtest.gamersnexus.net/news/intel-fights-back-arc-battlemage-xe2-gpus-changing-hyper-threading