191
Lemmy.ca & piefed.ca now behind anubis (lemmy.ca)
submitted 6 months ago by to c/main@lemmy.ca
44 comments fedilink hide all child comments
 

Thanks to a particularly annoying botnet, everyone's favorite anime cat girl firewall is now helping protect piefed.ca & lemmy.ca from bots and scrapers.

This is requests per second and these are all thousands of scrapers on residential IPs hammering us:

They'd increase their usage until the site started struggling, then move on. I banned their user agents, but have no interest in a cat & mouse game. Anubis should hopefully keep things running much smoother for everyone.

Let me know if you have any trouble!

all 44 comments

sorted by: hot top controversial new old
[–] [S] 48 points 6 months ago* (6 children)

  • red = obvious bots
  • blue = bots and users hitting the first anubis page (ie, it's 99.9% bots)
  • green = users.
  • permalink
  • fedilink
  • source
  • hideshow 6 child comments
    •
  • [–] 22 points 6 months ago* (2 children)

    I don’t go to this school, but I’m gonna check whether I’m caught in your new net!

    Edit: wew, I’m not a bot!

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
    • [–] 20 points 6 months ago

    Bots can get rekt. I am afraid it's still gonna be cat & mouse game, let's see how long anubis works for us (I also use it for my services).

  • permalink
  • fedilink
  • source
    •
    [–] 18 points 6 months ago*

    Yay!!

  • permalink
  • fedilink
  • source
    •
    [–] 16 points 6 months ago (1 child)

    F the bots. Would like to be able to have nice things. Happy that at least this is the 🇨🇦-made solution (at least the primary dev, anyways).

    Does Fedecan have the budget to throw a couple of bucks a month to Xe? Completely understand if not, I've done not-for-profit corps before and I know what it's like. But if the budget is there, spending it on a Canadian dev would be a nice choice, IMO.

  • permalink
  • fedilink
  • source
  • hideshow 1 child comment
    •
    • [–] 9 points 6 months ago (6 children)

    Name and shame. What are the useragent strings? Can the companies be identified?

    It won't affect me personally, because I already hate all AI companies. But maybe I could convince some people if I tell them what a specific company is doing.

  • permalink
  • fedilink
  • source
  • hideshow 6 child comments
    •
  • [–] 10 points 6 months ago (3 children)

    OP says residential IPs :/

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 7 points 6 months ago (1 child)

    https://stormproxies.com/ et al are the kinds of site that offer this. Backend accessible rotating residential IP addresses, makes finding the source of the scourge almost impossible

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
    • [–] 6 points 6 months ago (2 children)

    will I get anime cat girl challenges!?

  • permalink
  • fedilink
  • source
  • hideshow 2 child comments
    •
    • [–] 5 points 6 months ago (1 child)

    I'll be curious to know if y'all experience any federation issues. If not, I may introduce this on the mastodon instances I administrate!

  • permalink
  • fedilink
  • source
  • hideshow 1 child comment
    •
    • [–] 5 points 6 months ago

    Thank you!

  • permalink
  • fedilink
  • source
    •
    [–] 5 points 6 months ago* (last edited 6 months ago) (6 children)

    Is this why Thunderbird is suddenly spouting errors whenever it checks a lemmy.ca feed? I'm not a bot.

    EDIT: fixed

  • permalink
  • fedilink
  • source
  • hideshow 6 child comments
    •
  • [–] 4 points 6 months ago (2 children)

    Wait, Thunderbird queue with Activity Pub or Lemmy?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
    • [–] 6 points 6 months ago (4 children)

    Again. Not a cat.

  • permalink
  • fedilink
  • source
  • hideshow 4 child comments
    •
    • [–] 6 points 6 months ago (6 children)

    are there any alternatives to anubis?

  • permalink
  • fedilink
  • source
  • hideshow 6 child comments
    •
  • [–] 10 points 6 months ago (4 children)

    How come you're looking for an alternative? Does it not do the job for you or something?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 7 points 6 months ago (2 children)

    tbh i would prefer something silent instead of a full screen block page while it figures out whether I'm a bot or not

    I don't even like cloudflare click to confirm you're not a bot pages which auto confirm

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 9 points 6 months ago*

    To my knowledge, which is often wrong, that's necessary.

    It's a proof of work system, so your browser has to receive the challenge work, create background workers to do it, then submit the results and get authenticated.

    If the work wasn't challenging (slow), then it wouldn't be any impediment to scrapers and bots.

    Whether there are alternatives to proof of work that work well, I do not know. But fingerprinting alone is actually very difficult.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 3 points 6 months ago

    FWIW I think cloudflare and similar do the full screen thing too, they just render a blank page though so it just feels like more load time.

    I don't run Anubis on my stuff currently, but I'd be surprised if it doesn't have a similar feature

  • permalink
  • fedilink
  • source
  • parent
    • [–] 3 points 6 months ago

    Hell yeah! You guys are amazing!

  • permalink
  • fedilink
  • source
    •