you are viewing a single comment's thread
view the rest of the comments
[–] 6 points 1 day ago* (last edited 1 day ago)

I thought this would be nonsense at first, but the methodology is interesting. It seems plausible. And they tested quite a good spread of open weights, like MiMo and LongCat (which usually fly under the radar), all the way back to Yi 34B.

They missed Russia’s LLM, but that’s okay; it’s very obscure.

It’s also “lucky” they found this correlation, and so many recent weights happened to be initialized with PyTorch, on the GPU: very specific conditions.

…Also, I think the FP4 precision is a dead giveaway too, as Hopper wouldn’t be able to train that effectively, right? I assumed they were using at least some smuggled B200s.


And the export restrictions were stupid anyway.

  • source