And ethically trained. People should be financially compensated if their data was used to train AI without their consent.

It is, but the issue is the right time to buy hardware was 2 years ago.

I'm not surprised it's moving this direction, but I am surprised it's moving this direction now.

Hardware is extremely difficult to find right now and token prices are falling off a cliff in terms of price per intelligence. About 6 months ago I was talking about how it's only going to get cheaper from here and still seems to hold up today, we really don't need anything better than Opus 5.5 and compute is only going to keep increasing (price decreasing) who knows what the field will look like next year.

Either way I'm happy to live in a world pushing towards that, just surprised.

Microsoft predicted this some time ago which is why they have begun pushing a Hybrid Model where tasks are run locally first and only going to the cloud for heavy lift.

The data center is still needed, just not the same scale. You don't run "small" local models to compete with Claude's size. They don't publish numbers, but estimates were like 1-2T params for the models like early Opus. You need a lot of hardware to run those efficiently at a scale an enterprise would need.

Or they just spent enough to upgrade developer desktops with RTX Pro's to run quantized ~100B param models at their desks. Those are very good for coding.

You totally have a limit. It’s what the cards can output.

Revolut has 54x B200 cards (according to their video on this). So their limit is whatever that can output each month

I think the doomerism was all to get that moat. There's no vendor lockin either, even if you're using the API. You can switch between providers really easily.

Any sensible company, that has the captial, is hedging their bets. My place has plenty of Claude and Copilot licenses, but we also have healthy local projects going on to.

Personally I can use Claude if I want, but I tend to use the self-hosted platform (Qwen and GLM on DGX Sparks before anyone asks). Big Boss told everyone to be sensible with expenditure, so this part of me doing my bit.

Before anyone comments about "is Big Boss also saving money?". Yes he is - the guy comes from a French farming family, he is flies economy out of choice and is well-known in our industry for sacrificing not just his bonus, but also his salary when the company has been in rocky waters over the years.

It still makes sense for very small teams where the burden of maintaining the infra is too high, but if you have 2k engineers it's a no brainer to self host. It's only a matter of time before open weight models can process vision as well.

Yep, but obviously with a big initial investment.

Not saying it was ok, but the issue here was that the Italian police got hacked not revolut, from their pov the request was valid.

Cute that they called it $0.3 / 1 M tokens.

GPU WORLD
US$559,950 per unit: https://gpuworld.ai/gpu/nvidia-blackwell-b-series-gpus

So, IRL...about $3.5 million dollary-doos capex. Given they probably have thousands of engineers on staff, yeah, I can see it being worth it.

In the spirit of things, I asked GLM5.3 what that set up actually looks like. Ouch.

Fetched that page for real. Specs (from gpuworld's DGX B200 datasheet page):

Dimensions: 17.5" W × 87.5" D × 42.9" H (cm: 44.5 × 222.3 × 109.0)
Weight: 275 lb (124.7 kg)
Rack height: 10U equivalent
Power: up to 14.3 kW max per system

In real-world terms:

Height: 10U = 10 rack units = 10 × 1.75" = 17.5" tall — about the height of a bar fridge standing on its side. But that's the chassis; you'll also want ~2U of clearance for airflow/management, so plan for ~1.5U of breathing room above it in a rack.
Depth: 87.5cm = just under a metre — so it'll stick out of a standard 1m-deep server rack unless you've got deep racks (or you let the back panel hang open). Standard 42U racks are typically 1.1–1.2m deep to handle this.
Weight: 275 lb / 125kg. That's a hefty man's weight — two people minimum to lift, ideally four or a rack lift. Standard datacentre floor loading is usually fine at this density but worth checking if you're stacking multiple in a rack.

Rack math for 7 units: 70U of just these systems — so a single 42U rack can't hold them all (you'd need 2 racks), and at ~14kW each you're looking at ~100kW total draw — that's serious power/cooling, way beyond a standard office closet.

Short version: think "fridge-sized box, half a tonne with rails, eats 14kW" — not a pizza box.

Agreed. Economies of scale are about the only way this makes sense.

I was just working thru the use case for purchasing a 64gb Strix (vs the typical coding I might do using OR or Open Code Go). The ROI is somewhere in the order of 7-10 years for me. So, on a purely dollar cost basis, local inference is $$$. OTOH, from a sovereignty and "own your own code" basis...there's something to be said for it.

Probably I will run a few tests using OR with a few outstanding tickets of mine, to kick the tyres of Qwen3.8-27B. I've tried Qwen3.6-35B and frankly it's...not very good at code gen or patching.

To boot, code gen isn't the only thing I use LLM for, so I suppose it depends on how bloody minded I want to be about future use case (I do actually have one).

Ah...that's just the one-off costs. Remember...they still have to pay monstrous electric bills. Maybe in the order of $100K or more a month.

They are unsustainable, they always have been and likely will be until compute is a hundred times cheaper than it is now.

Yeah it really only works for a company of that size when you can tip the scale into buying or renting enough to run the compute, but it makes sense. And I’m sure that Nvidia and others are probably sitting on a goldmine of compute optimization they’re not releasing because they want to milk the cow for as long as they can, or keep the compute savings to themselves

Lol

until then, the american gov't is going to do everything in its power to force us to use the american models like they're doing with electric vehicles and other green tech rn.

Before you know it, companies and consumers will have the necessary hardware to handle all their AI needs with edge hardware (e.g. right on their phones and desktops—no need to call out to any servers).

i saw this too and it made me switch to graphene os because it and it makes me feel conflicted because i believe the anti-AI hysteria will fizzle out the same law the luddites did, but now i'm knee capping my own access wtih graphene. :P

We could have the equivalent of PhD-level knowledge on zillions of topics built into our PCs and phones and these hardware models wouldn’t need to call out to the cloud at all except to receive patches/updates that get applied whenever they run.

we've had access to this knowledge for the better part of the last 30 years and it's lead to bigger echo chambers with better propaganda -- the owners of these systems get to decided which pd-level knowledge they want you to have and all this asshattery will continue on as before as a result since it's the same people making the decisions for us right now.

Why, because of their slavic names? Or because you get a Lithuanian IBAN?

The company is headquartered in the UK and both founders have lived there for a long time, even if born elsewhere.

The IBANs are Lithuanian because that's where it was easier to get registered. Similar to how Wise got registered as a bank in Belgium despite being an UK business with Estonian founders and an office in Tallinn.