you are viewing a single comment's thread
view the rest of the comments
[–] [S] 4 points 1 year ago (2 children)

Seen plenty of people who think this is a bad thing, do they just want everything to be crawled. I mean I don't think this is the saviour but it has got to be better than wholesale theft

  • source
  • hideshow 2 child comments
  • [–] 5 points 1 year ago (1 child)

    do they just want everything to be crawled

    Yes. Web crawling has been a normal and vital part of the web from day 1. We'd have no search engines without crawlers.

    The web is user-centric by design. I'm sick of tech companies trying to flip the script and hoard information, most of which is not theirs to begin with (e.g. Google, Reddit, Twitter, Facebook, etc.).

  • source
  • parent
  • hideshow 1 child comment