This is depressing. Profoundly depressing. i look at the statistics board for my reverse-proxy and i never see less than 96.7% of requests classified as bots at any given moment. The web is filled with crap, bots that pretend to be real people to flood you. All of that because i want to have my little corner of the internet where i put my silly little code for other people to see.

From the conclusion, and I feel exactly the same looking at my server logs.

Ads make the web human-hostile and such disgraceful behaviors by scrapping bots force to make it bot-hostile too.

I am sad and depressed. I went into AI to solve the problems of the world and I still think that the progresses made in machine learning are a huge step to to improve the world, but seeing what capitalism turned these tools into... sigh.

I don't even have the strength anymore to explain that "AI companies" and "AI"are not the same thing...

There should be ways to behave correctly. Robot.txt should be legally enforced, rate limitation should be respected and prosecuted. Sites with information they are willing to share with models should just provide a datadump and individual requests should be reserved for human usage.

But an internet where everyone is understanding of each other and business actors do not act like psychopaths does not exist.

Self-hosting anything that is deemed "content" openly on the web in 2025 is a battle of attrition between you and forces who are able to buy tens of thousands of proxies to ruin your service for data they can resell.

This is depressing. Profoundly depressing.

Sigh

Good read, I use crowdsec to block most of the traffic considered as malicious (which tend to overlap with scrappers), but I should look into Locaine to feed garbage to the remaining ones instead of throttling.

I would like to know how well iocaines spanky new redirection module works compared to Anubis.

If nothing else, to see if throwing Anubis in front of iocaine is still a worthwhile idea.

Build your own captcha, there's just no other way to be sure it's human traffic with prebuilt solutions :(

Honesty we need a POW system built into http.