NASDAQ just crested a new high this month, before stumbling last week.

If the death of the tech industry is on the horizon, I can't see it.

You should not try to make sense of a decision they probably used chatGPT to make. It's bad for your metal health

The crazy thing is all that data is only good online.

For enterprise, I think same reasons why companies using paid software instead of open software. Someone else to maintain and blamed if something doesn't work.

training a top-level model takes harvesting petabytes / exabytes of revelant data and then months of continuous 100% power on a large server farm costing many millions of dollars.

The hardware requirements are substantial. If companies or individuals want to run models locally, especially at the speed you'd expect from the big name models, then everyone needs a pretty serious GPU on their work station. That or the company needs to build and maintain a server capable of running whatever loads they expect to see.

None of this is impossible of course. But you pay for the tokens one way or another. Either you pay the upfront cost of the hardware and continuing cost of the electricity to run it, or you pay with the inconvenience of running a model very slowly on lower end hardware, or you pay a subscription for someone else to worry about all that.

This is why local models are still niche. Only hobbyists nerds are willing to go through the trouble of setting up and running a local model. Most companies still are not sure if AI is actually worth it for them. So it's much easier to pay for tokens than the giant upfront cost required to build you own infrastructure.

If the numbers the AI companies are putting up as far as infrastructure cost vs revenue are anything to go by, it probably isn't worth it to most companies.

You need 200GB of video memory :)

Honestly, I have a little tin-foil-hat theory that the used their infine supply of venture to fuck the memory market to make sure people couldn't run giant chinese models on solar and eat their lunch.

They thought they sold tokens, they actually sell marginally (6 months lead) improved models.

They don't need to, OpenAI and Anthropic are running at a huge loss to keep it cheaper to use them than to buy the comparable hardware required to replace them. The only way running open source models is cheaper is if you ignore hardware costs, or if you have a very specific and unusual usage pattern (i.e. constant 100% capacity, and not a clock cycle more or less), or if you go with another cloud provider who are also running at a loss.

If you want it well rounded I can see the argument for enterprise but for me personally I programmed one on my workstation and it’s trained on very specific things I think it barely uses up all 24gb vram I have available.

Looks like the market is now trying to get out from under them.

If they weren't needlessly nervous (by that I mean paranoid), they would not be in that position. Whatever their wealth, those people cannot feel safe.

Based on the current valuations of their companies. They will likely remain 0.1%ers, but it will still hurt.