843
lads (lemmy.world)
submitted 1 year ago by to c/programmer_humor@programming.dev
57 comments fedilink hide all child comments
 
you are viewing a single comment's thread
view the rest of the comments
[–] 5 points 1 year ago (3 children)

But they can still scrape it, it just costs them computation?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 12 points 1 year ago* (2 children)

    Correct. Anubis' goal is to decrease the web traffic that hits the server, not to prevent scraping altogether. I should also clarify that this works because it costs the scrapers time with each request, not because it bogs down the CPU.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 1 year ago (1 child)

    Why not then just make it a setTimeout or something so that it doesn't nuke the CPU of old devices?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 1 year ago

    Crawlers don't have to follow conventions or specifications. If one has a setTimeout implementation that doesn't wait the specified amount of time and simply executes the callback immediately, it defeats the system. Proof-of-work is meant to ensure that it's impossible to get around the time factor because of computational inefficiency.

    Anubis is an emergency solution against the flood of scrapers deployed by massive AI companies. Everybody wishes it wasn't necessary.

  • permalink
  • fedilink
  • source
  • parent