Title.

I've noticed that the issues above are becoming increasingly notorious across the entirety of the Fediverse. What's being done to mititage those issues?

you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 4 months ago

> scraping puts considerably higher load on the server to the point where it brings down entire servers or at least severely degrades the performance for legitimate users.

Up to a certain point though. I think what most people are objecting to is the relentless nature of these AI crawlers, disrespecting robots.txt and requesting the same resources ad infinitum.

After all, search engine crawlers do the same, but at a more manageable scale.

The argument that it's "non-reciprocal" is valid, though.

  • source
  • parent