no comments (yet)

sorted by: hot top controversial new old
there doesn't seem to be anything here
8 comments in c/technology@lemmy.world view that post
[–] 15 points 1 day ago (3 children)

Here's the high level description of how it works.

  • Create a directory like /honeypot/ on your site
  • Put some code in index.php within the directory
  • Add the honeypot as an explicit "Disallow" in robots.txt
  • Include a hidden link to to the honeypot on every page of your site
  • If something accesses that page, write its IP address to your .htaccess
  • The .htaccess bans the IP address from ever accessing your site again.

But, I thought that scrapers often used proxy networks so they retrieve through huge amounts of IP's.

  • source
  • hideshow 3 child comments
  • [–] 2 points 1 day ago (1 child)

    Well, then these proxy networks will be banned. Maybe he should parse the ban lists occasionally if some entries can be compacted down to something/24 or so.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 3 points 23 hours ago

    The proxies are implemented by malware running inside TV sets, so the addresses are whatever Comcast or the like hands out. They're not organized in blocks, and there are millions of them. Typically you'd get just one or a few requests from each address.

  • source
  • parent
  • [–] 13 points 2 days ago (1 child)

    Sounds simple and effective, interesting! Is there also an option to send a missile strike to the originating location ?

  • source
  • hideshow 1 child comment
  • [–] 2 points 22 hours ago

    Alliteration always annoys

  • source
  • [–] 1 point 13 hours ago

    Have you considered using fail2ban instead of .htaccess and PHP?

    Like fail2ban crawls your access log, sees a request to /honeypot and adds the IP to its banlist.

    I have something similar running, if you try to login in my nextcloud too often you get fail2ban-ed too (instead of nextcloud disabling the user account which inconveniences me). Required forwarding the nextcloud logs to the nginx LXC but that works fine too...

  • source