115
How to combat large amounts of Ai scrapers (sh.itjust.works)
submitted 1 year ago by to c/selfhosted@lemmy.world
53 comments fedilink hide all child comments
 

everytime i check nginx logs its more scrapers then i can count and i could not find any good open source solutions

you are viewing a single comment's thread
view the rest of the comments
[–] 3 points 1 year ago*

I wonder why that RoboNope doesn't just make a fail2ban entry for anything that accesses a disallowed url and drop them entirely.

Actually this look like it would do something similiar, then dumps them to fail2ban after the re-access the honeypot page too many times: https://petermolnar.net/article/anti-ai-nepenthes-fail2ban/

  • permalink
  • fedilink
  • source
  • parent