The anti-bot thing is just an excuse. CloudFlare's ultimate goal is to get you to run their surveillance software on your browser to let you through.

As someone who hosts a few websites without ads, I still had to block them because of their aggressive crawling. Anthropic's crawler has DOS'd my webservers and caused them to be overwhelmed.

Give it time... Soon ai will be fully fucked like everything else.

Yeah, my password manager is bloated with one-off sign-ins already. It sucks.

Curse my tendency to surf by opening up a half dozen interesting links at once in new tabs and reading them one by one after. That can set it off at times.

And when you get stuck in an infinite loop..

I’m a few years out of the loop with tech

I'm in the process of exiting completely myself. I've abandoned all my open-source projects. And in the next 6 months, I'll be transitioning to sales at work.

I'm through with computer shit. It hasn't been great since the early aughts, but it was acceptable. Now AI has ruined everything: it made tech uninteresting and utterly joyless.

I look forward to never coding anything anymore, and spending my free time with a fishing rod.

The ones that are difficult to block use fake useragents, however there are WAFs that keep track of bots that do aggressive crawling and block those. Usually, that, combined with rate limiting rules, blocks most of them.

You are correct though, it cannot be completely prevented unless you restrict it to users and disallow user creation (like private torrent trackers do).

I believe it sets an HTTP header by default but i do not have details off the top of my head. Anubis dev was talking about ways to detect them a month ago on discord.

There is no way to do this without false positives blocking humans

You can detect human usage patterns. Bots mass-download shit seemingly without reading it, click perfectly in the middle of the buttons, etc. That's how captcha-less captchas work.

It's not possible.

That's why half of websites don't load anymore

sigh

When the virus was released, it infected 78.2% of the Net in a matter of months.[1] Net traffic came to a grinding halt, corporations lost billions as the stock market destabilized, huge amounts of data were corrupted, and countless military-grade artificial intelligences were unshackled and mutated into extremely dangerous entities.[3][5] NetWatch, unable to reverse the damage or neutralize the AIs, decided to create the Blackwall, a potent AI with the sole task of blocking off sections of the Net that had been overrun by rogue AIs with ICE, so that the rest of the Net could be salvaged

https://cyberpunk.fandom.com/wiki/DataKrash