you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 9 months ago (2 children)

Because the 1000 requests/10 minutes on my server are done by AmazonBot, mostly. Followed by ASNs from Huawei, Azure and the like.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 9 months ago

    AmazonBot follows robots.txt. I don't so what Huawei and Azure ASNs have to do with it - that sounds like those requests simply come from inside a Huawei and an Azure network, respectively, but could otherwise be anything.

  • source
  • parent