[–] [S] 11 points 1 year ago (7 children)

I've expanded the cloudflare challenges to all non Australian requests. This has made a massive impact.. I'll look at ways to relax this tonight.

For now users outside Australia will receive a cloudflare challenge in their browser. Lemmy apps will likely not function.

  • source
  • [–] 8 points 1 year ago (2 children)

    Possibly. It'd be a terrible way to get that data if so. You could just spin up a Lemmy instance and federate with all of Lemmy far easier than trying to scrape all the web front-ends. Also, we try to fly under the radar from the Internet a bit. We opt out of Google searches, Amazon, Apple and GPTBot for example.

    While our data is all human-generated (attractive), we're a pretty small userbase. There are shinier web sites to scrape than us.

  • source
  • parent
  • context
  • [–] 4 points 1 year ago* (last edited 1 year ago)
    view more: ‹ prev