If they could stop blocking real users at the same time they block AI crawlers, that would be nice.

A nice explanation of what's wrong with web-based AI. I hope other content providers follow suit.

Kind of reminds me of that time CF bait and switched that gambling website. Everyone was wondering who they find more predatory and distasteful.

Cloudflare aren't perfect, but I still use them because for a free account the benefits outweigh the negatives like this.... However to say the worlds leading publishers and AI are on board is simply not true...

This has been tried before many times. The problem is that this exchange can never satisfy all of the parties.

Would site host take 0.01$ for a page? If its Walmart.com then they'd happily lose even 0.10$ or more if competitors can't analyze their products and other perceived IP damages.

For example, let's assume they do the business math and come out that if it is anything below 5$ is a no deal - what scraper would pay 5$ for a single product page scrape? Maybe openAI can pay that but is this what we want where public scraping is only accessible to billionaires? What if you're just a user that wants to track Walmart price to build your own budgeting script? Are you paying 5$ on every request?

Now for creative content like blogs etc. it could actually work and micropayments have been holy grail here forever but what more likely to happen is that free content will outcompete paid because when LLM asks do you want to read this for free or pay 2$ for this other source 99% of the users will pick free because some unknown source has zero authority in the end user's eyes to justify this risk.

What I suspect will happen is similar with what happened with SEO spam rise but it'll be a but better because LLMs are harder to game than Google. Most content will be free but have injected biases, shilling or other promotions or agendas to subsidize the costs. On the other hand a lot of content will remain high quality and free as a legitimate source for authority signaling within relevant industries which is already a big thing.

Anyway thanks for coming to my ted talk

Could this conflict with the fair use of AI training rules?

Awesome, clear, you know how to do it, you can do it!

2010-ish: copying is not theft, stealing a thing leaves one less left, copying it makes one thing more, that's what copying's for... 🎵🎶🎵🏴‍☠️🏴‍☠️🏴‍☠️

2025:

I'm myself not overly enthusiastic about AI, not at all. I think LLMs are mostly bullshit generators with very little practical use. But from a public policy point of view, the stance that training AI should require consent of or payment to the copyright holders is IMHO wrong; if we accept it, what is next? Humans summarizing and paraphrasing sources (e.g. for a wiki article) should require consent/payment? Humans learning from sources should require consent/payment? I think wanting a free and open society means taking the side of AI operators, not copyright holders, in this matter.