I am not that concerned with the traffic, but I want to dissuade them from adding my texts to their corpus if possible. More out of principles than out of any illusion that it has any real, practical consequences.
Currently my plan is to serve a restrictive robots.txt (which I assume is completely ignored), pass traffic through Anubis with a policy that allows regular browsers without challenge but denies known scrapers (I don't really think they send truthful user agent strings so this probably won't do much either), and configure Nginx to be more aggressive with rate limiting. I also plan to license texts under a non-commercial CC license (which I also don't think will really prevent them).
I wonder if anyone have experience with such setup and can report how much or little this actually does anything?
The full, more interactive and JS-enabled site will issue challenges though.