Scraping is neither new, nor always malicious. Without scraping, no search engine would work and there would be no archive.org wayback machine.

However, AI scrapers all copy the same shit over and over again and do not intend to lead traffic to your site. They just cause cost and don’t give anything in return. This is the problem.

The web is for user agents

That's entirely too much work when they can just make one vibe-coded bot and just offload the scraping costs to us as externalities

Yes, but that's work and costing the scrapers resources, while simply siccing some scraper is simpler and costs less.

Proof of work means that your client has to do some "work" in order to gain access. It typically means a challenge that can't be trivially solved, but can be trivially verified.

For example, the challenge may be something to the effect of:

"Give me a string, that when hashed by md5, results in a hash that ends in 1234".

Your browser can then start bruteforcing until it finds a string (should take a few seconds max), and then it can pass the string back to the server. The server can verify with a single hash, and you're in.

Its not wildly different to crypto mining, but the difficulty is much lower for antibot, as it needs to be solveable in seconds by even low end devices.

https://en.wikipedia.org/wiki/Hashcash

You gotta do some math problems before you're allowed to play outside.

But for computer.

It shouldn't. So long as you can run JavaScript, it should work everywhere.

Most clients use an API connection which is specific to Lemmy. This is extra work to make scrapers speak that language so no scraper does.

For the moment, a scraper trying to hit any API endpoint would just get a simple malformed request error. With any luck, it stays this way so we don't have to protect API directly.

I may be misunderstanding this measure but I don't think that's going to be mitigated.

If I understand correctly, this requires browsers requesting a page to do a small amount of "work" for no reason other than demonstrating they're willing to do it. As a once off for devices used by humans, it's barely noticeable. For bots reading millions of pages it's untenable - they'll just move on to easier targets.

However, that only works for bots who's purpose is to injest large quantities of text.

For a bot who's purpose is to make posts, or upvote things, or reply to other comments, they're much less sensitive to this measure because they don't need to harvest millions of pages.

I would, however. I don't know if electricity with the correct voltage and amperage just grows in trees up there in the US, but in the rest of the world, we have to pay up for electricity, and having to consume more of it also means a larger damage to our local environment, already preyed upon by northern-hemisphere corporations.

Not to mention, it effectively raises our power bill for no new gain as well, which comes with a very bad timing due to recent scandals (up to Constitutional Accusation Summons) in how the costs of energy transportation are being billed to users in my country. Besides all the local increases of cost, it mechanically functions not very different from rent-seeking.