Anubis rarely ever annoys me (I still prefer it over cloudflare's Mitm) but this specific visit feels almost vindictive. It used half my CPU cores at full blast for three straight minutes.

It seems to be effective since I can't really "bypass" it with extensions, so AI crawlers probably can't either, but damn there really should be an option to complete a manual captcha or email the administrators to complain.

As I'm typing this the challenge is still running xD

all 64 comments

sorted by: hot top controversial new old
[–] 147 points 3 months ago (4 children)

I wish it were doing protein folding work or something less wasteful.

  • source
  • hideshow 4 child comments
  • [–] 52 points 3 months ago* (3 children)

    I wonder if that could be a thing

    FoldCoin

    put crypto nonsense to work and maybe help fund research

  • source
  • parent
  • hideshow 3 child comments
  • [–] 58 points 3 months ago

    Unfortunately yeah, people with exceptionally shit computers get caught in the cross-fire of these things

  • source
  • [–] 44 points 3 months ago (1 child)

    I've had Anubis get stuck for me before. Usually clears up if you refresh the page

  • source
  • hideshow 1 child comment
  • [–] 32 points 3 months ago (2 children)

    A word of advice: download Arch wiki for offline use.

    You can use Kiwix to have a full graphical wiki, or use things like arch-wiki-docs.

  • source
  • hideshow 2 child comments
  • [–] 32 points 3 months ago

    I wish AI scrapes didn't make the Internet unusable to the point of them using Cloudflare MITM or Anubis

  • source
  • [–] 40 points 3 months ago* (3 children)

    Think of it like tipping culture. Get mad at AI industry scrapers that make anubis necessary. Sam Altman is the reason you are enduring this. Take a lighter and...

  • source
  • hideshow 3 child comments
  • [+] -6 points 3 months ago (1 child)

    I suspect this is more about all the individual searches than mass scraping. We've had google et al scraping the entire web for a long time and nobody gave a fuck. AI makes it easy to launch a ton of web searches without putting much thought into it.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 3 months ago*

    Search engines doing it is good because now you are indexed and will show up in searched and they won't do it as often as AIs.

    AIs will take your stuff, DDOS you in the meantime and do it periodically - by different people too, they don't give back anything either

  • source
  • parent
  • [–] 15 points 3 months ago

    Difficulty 6 is a little high. The default is 4.

  • source
  • [–] 12 points 3 months ago

    You could try hitting up the Internet archive for a quick workaround.

    I think you can also download a copy of the entire arch wiki as a zim file too

  • source
  • [–] 15 points 3 months ago (5 children)

    The idea is that at individual scales the additional load is ignorable

    So uh anyone anyone gonna tell them the idea was wrong? Three minutes is not ignorable.

  • source
  • hideshow 5 child comments
  • [–] 18 points 3 months ago* (4 children)

    The difficulty is typically dynamic based on current request load. At this difficulty typically means their server is getting hammered really hard, better this than the server just respond with a bad gateway error.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 0 points 3 months ago (3 children)

    It would be better to serve a static page saying "the server is getting hammered really hard", and, if possible, context about the duration of the current event and historical frequency.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 3 points 3 months ago

    Less sites are doing that to intentionally avoid tipping off the AIs about thresholds and trigger behaviours. The two main solutions are browser crypto challenges like this, or silently diverting flagged connections to a tar-pit of fake pages. The former is easier to implement but the latter actively sabotages the scraped data.

  • source
  • parent
  • [–] 24 points 3 months ago (7 children)

    Wait, I'm not mining crypto trash for strangers whenever I see that annoying fox girl, am I?

  • source
  • hideshow 7 child comments
  • [–] 38 points 3 months ago* (last edited 3 months ago) (4 children)

    Proof-of-work in approach, not towards the same ends. The goal is to prevent scraping, not monetization.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 4 points 3 months ago (3 children)
  • [–] 4 points 3 months ago*

    Also this instance; can't browse it on Desktop with a Librewolf-alike policies.json. Can the sensibility be adjusted server-side? No problems on Archwiki & co.

  • source
  • [–] 4 points 3 months ago*

    I've had websites like Nix Hydra randomly give me a webpage full of gibberish which is also annoying and a lot moreso than the Anibus ones because you can't get past it, I thought they got hacked at first also

  • source
  • [–] 4 points 3 months ago (7 children)

    I wonder if this even is effective. If a bot can solve anubis challenges, why can't it solve all difficulties?

  • source
  • hideshow 7 child comments
  • [–] 15 points 3 months ago (4 children)

    It's not about stopping the bots, it's about slowing them down

  • source
  • parent
  • hideshow 4 child comments
  • [–] 2 points 3 months ago* (3 children)

    But once the challenge is completed, they have the token and can scrape until it's expired right? So if it's just taking them a bit more time to get the auth token, what is it solving?

    Example: the token lasts for a week and it takes 30 seconds to get the token.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 5 points 3 months ago* (last edited 3 months ago) (1 child)

    I would assume the token is only good for a few page accesses.

    E.g. 500 would mean that a person would see this extremely rarely, and still block a bot from crashing the site.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 4 points 3 months ago*

    Wild that I thought a difficulty of 4 was reasonable when I set up Anubis on my reverse proxy.

  • source
  • [–] 4 points 3 months ago

    the most I'm giving any site is 6 seconds on a typical visit, 10 if I'm feeling generous. if I'm not seeing shit at 10 your site is broken

  • source
  • [–] 3 points 3 months ago

    yeah this shit takes so much longer on old hardware and nobody gives a fuck

  • source
  • [+] 5 points 3 months ago (4 children)
  • [–] 3 points 3 months ago (5 children)

    I've actually wanted to deploy this with some absurdly difficult challenge as a troll but I don't know what's worth putting behind a webpage with effectively a client side loading bar lol.

  • source
  • hideshow 5 child comments
  • [–] 2 points 3 months ago

    Good to see it's not just me. I kept receiving heat (pun intended) whenever I criticized these "Proof-of-work anti-bot solutions", now I finally know I'm not alone.Like an unskippable Youtube ad, we're supposed to sit three minutes while these "solutions" dangerously burn our CPU regardless of whether it's an old CPU, and (!!!). At least Youtube ads don't (usually) burn the CPU the same way.And, no, it's nowhere close to being infuriating to the supposedly intended targets of Anubis. OpenAI and the alike are likely laughing of Xe Iaso while they all burn more coal and increase the hyperscale data centers, with Anubis being just a pesky, insignificant pebble inside businessmen shoes. Meanwhile, we, the human users caught in this crossfire, including me with my poor decade-old Acer ES1-572 laptop, we can't afford a new machine should Anubis, Cloudflare and other similar "anti-bot solutions" end up "weighting" our CPU too much it finally makes its way to the underworld with no one being legally liable for the physical damages caused by such algorithms.If only Anubis, supposedly "different" from Cloudflare's corporate nature, had a "Proceed or Leave" button or checkbox to start the "weighting your heart" thing, I wouldn't be complaining or actively websites using "PoW anti-bot solutions" (I made myself a Firemonkey userscript to speedrun a window.close() for pages whose title is one of the "Just a moment..." or "Making sure you're not a bot!", because neither Xe Iaso nor Matthew Prince nor the webmasters using their "PoW anti-bot solutions" would buy me a new laptop if mine eventually died from facing heavy cryptographic computing while my CPU was already struggling with a Blender rendering process in the background). But, no, it's no different than the corporate they swear to be fighting against ("something something one who fights with monsters should be careful lest they thereby become a monster themselves")... !mildlyinfuriating@lemmy.world

    Meme "they're the same picture" with Anubis PoW Captcha page and Cloudflare PoW Captcha page.

  • source
  • [–] 1 point 3 months ago (6 children)

    Every time I watch Youtuße through Invidious and I get this crap, it gets stuck every time and goes nowhere.

  • source
  • hideshow 6 child comments
  • [–] 3 points 3 months ago (3 children)

    Public invidious instances are more a proof of concept, its much more performant to run it yourself.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 1 point 3 months ago* (last edited 3 months ago) (2 children)

    True.

    Will set one up, if more need arises.

    ...but people keep sharing invidious links.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 3 points 3 months ago (1 child)

    The last part of the url is still the same Video id from youtube. You can copy that in whichever you use.

    The internet actually really needs an overhaul to how links are presented. So you can press a video link and opening in the compatible player you prefer.

    Same issue exists on lemmy. A link will send you to the instance of the poster, we have special ! Links to bridge the gap but i can never remember how to use em.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 2 points 3 months ago

    Technically, the internet already has what you're looking for. It's just not really standardized for video media.

    The beginning of the URL or the schema is already able to on most mainstream OS be controlled by a specific application.

    There's nothing really stopping anyone from making it so there's a video:// schema, and then configuring a schema handler to handle it. on Debian it would be a single desktop file with a MimeType=x-scheme-handler/video line indicating that type of url should be opened with that desktop file. I had to configure it for vortex to get it to open mod links, I never knew how easy that was

  • source
  • parent
  • [–] 2 points 3 months ago (1 child)

    This is every scraper for YouTube.

    The only applications that I've ever seen reliably access YouTube are the ones that work by patching the YouTube app and then using what the app would do.

    Applications like invidious or Grayjay almost always have a 15 to 30 second delay before any video.

  • source
  • parent
  • hideshow 1 child comment