everytime i check nginx logs its more scrapers then i can count and i could not find any good open source solutions

you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 1 year ago* (2 children)

What's bothering you?

  • Is it to give out data for AI training? I guess you can't fundamentally protect against this, except by limiting how much content is provided to each address.
  • Or is it the resource strain that it causes on your server? In that case i recommend limiting how much a single client / IP address can request in a day.
  • source
  • hideshow 2 child comments