I much prefer to see Anubis rather than some bullshit captcha with a grid of AI generated slop that requires 30 clicks to pass.
post
I don't mind the second it takes. Better than the service going down because of AI bots.
Actually the opposite for me - I love to see her for a second, brightens the day a bit, fights the evil megacorps.
Can take a fair bit longer on low end devices. Don't have an issue with it beyond that
Anubis is open source, self-hosted, doesn't block me just because I use a VPN and the later versions work even with JavaScript disabled!
Fuck Cloudflare, long live Anubis!
How exactly is a proof of work engine suppose to run, without any JS work?
https://anubis.techaro.lol/docs/admin/configuration/challenges/metarefresh/
Apparently it just tells your browser to refresh after x seconds.
That's just protection by obscurity then. Any targeted attack could do that challenge at zero cost.
Seems utterly pointless though...
With the proof of work approach, at least it's demanding the client consume some resources, though the 'right' amount is a tricky question, either it's so trivial as to hardly matter to the scrapers, or it's hard enough to put a dent in the scrapers' build, but human operated low end devices are royally screwed..
Here the crawler simply schedules a resumption and moves on to other work. The crawler doesn't need it right now and it's free for it to wait.
Which doesn't work with some browsers, but it's acceptable, I guess
Cloudflare is great compared to Recaptcha.
Anubis is the best by far.
I love how toxic she is to corporate professionalism.
Its also perfect marketing, the software is free with the mascot hardcoded in. The official way to change it is to contribute to get an enterprise version.
The code is MIT licensed, what's preventing you from just removing the logo/changing it with something else....
Her face is the response to years of enshittification; without her, the modern browsing experience would suck much harder. Glory to Anubis!
Who is she supposed to be?
Anubis - an anti-scraping plugin for websites. I believe one of its claims to fame is placing LLM crawlers into "tar pits" (preventing them from eating website resources)
It does show up whenever you load into the site though, so I guess that makes their mascot stick in everyone's head.
Still better than this

Both have different purposes.
The Anubis challenge could be easily and cheapely solved by any JavaScript engine. It only becomes expensive for a massive number of petitions.
If for instance you would want to register a few thousand emails in a forum anubis is not going to stop anyone.
In fact I'm sceptical about really having an impact. As even when the challenge goes up in difficulty is not that expensive compared with all other cost related to these kinds of attacks or massive scrapes.
My suspicion is that most websites using Anubis see a positive impact because most crawlers and probers doesn't take into account Anubis, so they don't even attach a way to solve the challenge and they directly go into the "rejected by anubis" bucket. But any targeted attack I suppose would pass easily, either by doing a slow attack not to up the challenge very much, or just eating the cost. Imagine an AI company that using nuclear plants for training data, the cost of solving a few million JavaScript challenges is nothing in comparison.
As a DDOS mitigation it helps, but once again it's just a matter of eating the cost by the attacker. And the attack will still deny some service as the challenge go up and new legit users would also need to solve harder challenges.
I'm on discuss.tchncs.de right now and its web UI lags badly probably due to a lot of bot scaping.
If we had Anubis i guess that could be avoided.
Thank god we have anubis
Long live the Canadian anime catgirl!
What am I missing to understand this? What is Anubis?
I've seen this briefly pop up while looking up linux stuff online recently. Wondered what it was, thank you Lemmy community for some enlightenment!
I am skeptical about the real level of protection that Anubis really provides.
At the end is an automated test. Meaning that any machine could easily solve it.
Most "attackers" wont bother solving it because they don't really care. But if they would want they could. It's sort of protection by obscurity.
The more Anubis it's used the more we see attacks that actually equip a way to solve the challenges. Then is when Anubis up the challenge and the battle begin, between how much can Anubis up the challenge so normal users can still browse and how much cost the attacker is willing to eat.
Giving that these attackers tend to have high budgets I'm not that certain about its actual capabilities to reject a targeted ddos.
As for crawling for big data. I do think that it does nothing here. Companies willing yo scrape big amounts of data, for AI training or other purposes, have massive budgets and the electricity cost of solving the JavaScript challenges become nothing in comparison. They also doesn't need ro deny the service so they could spread the scrape to keep the challenge low reducing the cost even more.
Once again, positive results we currently see in practice I believe that are caused just because most scrappers and ddos attackers are just blindly attacking and doesn't really equip themselves for Anubis. Protection by obscurity. But a well equiped attacker I don't think it would have that much trouble getting past it, specially for scrapping, or other type of bot attacks that could be slowed down.
You're right, although my understanding is that there are a lot of poorly implemented scrapers for AI services unintentionally DDoSing websites with requests, so Anubis is more of a mitigation against those.
Yeah, seems like the problem is that fundamentally it could work by upping the difficulty a smidge making it then meaningfully expensive, but the spread between slowest edge device and high end means it's impossible to chase that difficulty without screwing over low end device users..
Yet Anna's Archive still doesn't use it, curious.
TBF Anna's just provides a dataset for people who want it and don't directly host any of the content (they're just the index). Anubis probably isn't as necessary for them.
Unfortunately, yes it does. Unless I want it to collapse regularly.
it kinda does when the web is being scraped by bots to fuel the LLMs
Banning crawlers doesn't have to affect UX for humans. And you can whitelist your favorite crawlers too if that's your thing.
If you're going to increase my compute to access your site, at least give me a crypto token that may or may not be worth anything

top 66 comments