everytime i check nginx logs its more scrapers then i can count and i could not find any good open source solutions

you are viewing a single comment's thread
view the rest of the comments
[–] 6 points 1 year ago

If you're able to, use GeoIP ranges to only allow access from the countries you want.

That immediately limits a lot of everything

Then - again if you're able to - use a block list that covers known scrapers in case they're in your country.

I use pfBlockerNG on my pfSense firewall for exactly this.

  • source