LLM scrapers and bots are a continuing challenge. Lemmy is not behind Cloudflare and custom filtering rules using NGINX only gets you so far.

I'm trying the setting called "Private Instance".

It's my understanding that all this does is require a user to be logged in to use the instance. This will hopefully cut down or stop the constant scraping. Preliminary signs are that it helps a ton.

  • walden
you are viewing a single comment's thread
view the rest of the comments
[โ€“] 5 points 2 weeks ago

Another vote here for anubis! It cut our traffic down by some 75% or so.

  • source
  • parent