everytime i check nginx logs its more scrapers then i can count and i could not find any good open source solutions

you are viewing a single comment's thread
view the rest of the comments
[–] 14 points 1 year ago (1 child)

Wern't there a few AI maze projects in the works? I wonder if running one of those for a bit will cause you to be added to an ignore list, clearly they dont respect your robots file.

  • source
  • hideshow 1 child comment