A lot of article websites whitelist Google's crawler, right? Can't that be done on a domain-basis? Or maybe have some kind of authentication-based system 🤔, I guess that would require the crawlers to be refactored though
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments