GPU-unlock: FP/BF16 instruction issue rate on Nvidia 3000's cards : hackernews@lemmy.bestiver.se

GPU-unlock: FP/BF16 instruction issue rate on Nvidia 3000's cards (github.com)

submitted 2 weeks ago by rssbot@lemmy.bestiver.se [B] to c/hackernews@lemmy.bestiver.se

comment fedilink hide all child comments

no comments (yet)

sorted by: hot top controversial new old

there doesn't seem to be anything here

19 comments in c/localllama@sh.itjust.works view that post

[–] mindbleach@sh.itjust.works 3 points 3 weeks ago (14 children)

Soft-locking hardware is a fucking crime. It's anti-competitive dumping.

hideshow 14 child comments

[–] Ajen@sh.itjust.works 3 points 3 weeks ago* (last edited 3 weeks ago) (11 children)

I'm split... A lot of the time it's cheaper to make the low-end version of a product by locking down the high end version, due to economy of scale. If they were forced to use lower end hardware, some cheap products will end up costing more to manufacture.

I definitely think our right to tinker with stuff and unlock features should be protected, though. Along with the right to repair.

parent

hideshow 11 child comments

[–] mindbleach@sh.itjust.works 3 points 3 weeks ago* (last edited 3 weeks ago) (10 children)

That's what this bullshit is faking, to price out products which are genuinely cheaper.

They spent the full cost for the high-end product, but you don't get to use it - because anchor framing lets them really gouge people for a marginally better product. Or the same goddamn product without those artificial restraints.

parent

hideshow 10 child comments

[–] Ajen@sh.itjust.works 1 point 3 weeks ago* (last edited 3 weeks ago) (9 children)

The reality of mass manufacturing means that, at least in some cases, it actually costs more to create a second factory line than it does to build the low end version with the the same internal hardware as the high end version. This is especially true with silicon, the vast majority of the cost is in design, test, verification, and building the fab. Once they're manufacturing chips, each additional chip doesn't cost all that much to produce.

parent

hideshow 9 child comments

[–] mindbleach@sh.itjust.works 3 points 3 weeks ago (8 children)

Which is why... pretending you have a low-end and a high-end... is fraud.

If they had one price for that one product, other companies with other factories could offer difference price-for-value options. That competition drives prices down and value up. But it is actively prevented when the complex business strategy of lying lets them make things objectively worse on purpose, to squat across an arbitrary portion of the market, and make up whatever damn prices they want.

parent

hideshow 8 child comments

[–] Ajen@sh.itjust.works 2 points 3 weeks ago (7 children)

What do you think would happen if it was illegal? I'm only coming up with two scenarios:

The cost of low end tech goes up because it costs more to manufacture
Low end products are discontinued because they can't make a profit

parent

hideshow 7 child comments

[–] mindbleach@sh.itjust.works 2 points 3 weeks ago (6 children)

I just fucking explained how competition does the opposite of that.

parent

hideshow 6 child comments

[–] Ajen@sh.itjust.works 1 point 3 weeks ago (5 children)

If it costs more to manufacture a product than it can be sold for, it won't be manufactured. Competition isn't going to drive down the manufacturing cost, it only drives down sales price. If you drive the sales price too far down, or drive the manufacturing cost too far up (or a combination of the two), the product gets canceled.

parent

hideshow 5 child comments

[–] mindbleach@sh.itjust.works 1 point 3 weeks ago (4 children)

You're already paying the lower price... for the full-featured object. That's the whole topic! If you can't imagine a competitor offering a better deal that customers prefer, what are you worried about?

parent

hideshow 4 child comments

[–] Ajen@sh.itjust.works 1 point 3 weeks ago (3 children)

If a company sells one product at 2 different prices, everyone will pay the lower price. I've already explained how it's cheaper to manufacture a low-end product based on a high-end design instead of creating a totally new design.

Do you think you'd be better off if companies were forced to remove unused components from low-end versions of products? How do you think that would benefit you?

parent

hideshow 3 child comments

[–] mindbleach@sh.itjust.works 1 point 3 weeks ago (2 children)

Baloney. You stated the rationale behind selling the same high-end internals as a low-end product, as if I didn't know. As if that's not the complaint.

I am telling you for the fourth time - that means there is no low-end version. It's all the same goddamn product. The separate price points are artificial, arbitrary, and a nakedly anti-competitive practice. It is a fraud which destroys the incentives behind tolerable prices for optimal quality.

What benefits everybody is when multiple companies want your money, and will balance their shameless greed against your ability to buy from someone else.

parent

hideshow 2 child comments

[–] Ajen@sh.itjust.works 0 points 3 weeks ago (1 child)

This conversation is going nowhere. I think I've adequately explained my point. Peace.

parent

hideshow 1 child comment

[–] mindbleach@sh.itjust.works 1 point 3 weeks ago

You weren't even listening to yourself.

parent

[–] AsAnAILanguageModel@sh.itjust.works 1 point 3 weeks ago (1 child)

On the other hand, if it can be soft unlocked, that’s pretty nice.

parent

hideshow 1 child comment

[–] mindbleach@sh.itjust.works 1 point 3 weeks ago

It certainly beats AMD deliberately burning cores to maintain stratified pricing.

That example starts well enough - they were triaging failed four-core dies into three-core dies. But as yields predictably improved, they kept degrading some fraction of perfectly functional processors, because those were selling quite well, and they didn't want to hurt the fat margins on the quad-core version. Even if they were all quad-core versions.

parent

[–] mal3oon@lemmy.world 1 point 2 weeks ago

This is very interesting for me! Thanks for sharing!

[–] BeefAndPoultry@lemmus.org 1 point 3 weeks ago (2 children)

I didn’t write down my exact tokens/s in llama.cpp before installing the unlocker, but the numbers I got after were roughly the same as I remembered, which also makes sense since I use quantized models that afaik use integer tensor ops.

token generation is bottlenecked by memory speed not compute power. your prompt processing/prefill speeds might've gone up

hideshow 2 child comments

[–] fhein@lemmy.world [S] 1 point 3 weeks ago (1 child)

Might be more relevant for LoRA training, right?

parent

hideshow 1 child comment

[–] BeefAndPoultry@lemmus.org 1 point 2 weeks ago

yea I would guess LoRA training uses bf16

parent

2 comments in c/stable_diffusion@lemmy.dbzer0.com view that post

[–] tal@lemmy.today 2 points 3 weeks ago (1 child)

The install procedure does involve changing some kernel boot parameters, so if you make a mistake it's possible that you soft-brick your OS.

I mean...just interrupt the boot process at GRUB, and edit the kernel boot parameters to whatever you want, boot into the OS, and then change them back persistently.

hideshow 1 child comment

[–] fhein@lemmy.world [S] 3 points 3 weeks ago

Yes, it's absolutely recoverable if you know your way around the OS, but now that Linux is getting more popular I suspect there are an increasing number of people who wouldn't know how. The reason I didn't write the exact commands I had to run to activate the unlocker was that I wanted to reduce the risk of someone copy-pasting commands they don't understand into a terminal, and then being unable to repair their install if it stops booting.

parent