[–] 5 points 2 months ago

Honestly, the character is the worst part about it, but that can be changed with an enterprise license

The Javascript requirement is not ideal, but the reality is that the vast majority of the internet already needs it, so I don't see it as that much of a hurdle. The other problem it has is that it requires a sort of herd immunity to achieve the goal. If a critical mass of websites don't use it, then the AI companies can simply just pay for the increased compute costs for the sites that do.

In a world where more than half of websites that are being scraped had some kind of protection like this, it would work much better.

  • source
  • parent
  • context
  • [–] 9 points 2 months ago (5 children)

    This is a good point. Honestly, I think the tac that Anubis takes is probably the best one right now.

    If each request you make to a web server is computationally expensive, that makes it harder for the bots to get that info at high speed, and thus needs to spend more power to access it. If a solution like that is deployed at a wide enough scale, it becomes prohibitively expensive to do the scraping.

  • source
  • [–] 3 points 3 months ago* (2 children)

    This seems like it'd be pretty good for obfuscating email addresses from crawlers. At small point sizes the real text is extremely obvious to a human, but both LLM OCR and pytesseract was failing to extract the real text from this image

  • source
  • view more: next ›