everytime i check nginx logs its more scrapers then i can count and i could not find any good open source solutions

you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 1 year ago (1 child)

Is ignoring robots.txt considered "honest"?

  • source
  • parent
  • hideshow 1 child comment