you are viewing a single comment's thread
view the rest of the comments
[–] 7 points 1 year ago (6 children)

lol. AI firms stop reading robots.txt

  • source
  • hideshow 6 child comments
  • [–] [S] 5 points 1 year ago (5 children)

    I think the idea is that all parties would find it beneficial so they would be incentivized to read it:

    Leeds told Ars that the RSL standard doesn't just benefit publishers, though. It also solves a problem for AI companies, which have complained in litigation over AI scraping that there is no effective way to license content across the web.

  • source
  • parent
  • hideshow 5 child comments
  • [–] 5 points 1 year ago (4 children)

    I imagine that when it comes to scraping data, the benefits of having the data far out away the risk of getting sued for scraping it.

  • source
  • parent
  • hideshow 4 child comments