14
[Meta] Lemmy (not PieFed) changed to a Private Instance (sub.wetshaving.social)
submitted 2 weeks ago by to c/wetshaving@sub.wetshaving.social
7 comments fedilink hide all child comments
 

LLM scrapers and bots are a continuing challenge. Lemmy is not behind Cloudflare and custom filtering rules using NGINX only gets you so far.

I'm trying the setting called "Private Instance".

It's my understanding that all this does is require a user to be logged in to use the instance. This will hopefully cut down or stop the constant scraping. Preliminary signs are that it helps a ton.

  • walden

all 7 comments

sorted by: hot top controversial new old
[โ€“] 4 points 2 weeks ago (4 children)

Do you want to give Anubis a try? That has helped our instances avoid the bots

  • permalink
  • fedilink
  • source
  • hideshow 4 child comments
    •
  • [โ€“] [S] 4 points 2 weeks ago (2 children)

    I've heard mostly good things about Anubis, but I've also heard it can be hard to get it right. I've never looked into it so I'm not sure what people are referring to... maybe the balance between blocking lots of bots but still letting humans through?

    This Lemmy instance is mostly sticking around for posterity, so hopefully Private Instance cuts down on the B.S. so I don't have to think about it any further!

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [โ€“] 3 points 2 weeks ago (1 child)

    I had some challenges setting it up which were mostly related to using load balancing for Lemmy-UI. In general there isn't much of a problem letting humans through so long as you only apply it to the front end UI not to the API.

    We have a rule to allow anyone with a jwt set which means logged in users are not challenged.

    Finding an instance that blocks content from logged out users would likely put off new users, but if you're not looking for new users then requiring login seems like an easy solution to the problem.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
    • [โ€“] [S] 3 points 2 weeks ago*

    @otter@lemmy.ca @hal@piefed.nz thanks for the encouragement to try Anubis.

    The instance is now running using the Ansible playbook, with Anubis enabled. I changed the server back to non-private (which didn't help, anyway).

    I've been interested in switching to running Lemmy via the Ansible playbook for a long time, but now that I've done it I miss "my way"! I'll either have to go back to "my way" or fork the playbook and learn how to make ansible do things how I like it.

    Mostly, I need to figure out how to have everything go through Nginx Proxy Manager instead of the built in nginx container. I like having everything on an nginx docker network so they can use hostnames to find each other instead of IP addresses.

  • permalink
  • fedilink
  • source
    •
    [โ€“] 2 points 2 weeks ago

    dbzer0 is using haproxy with haphash successfully to mitigate.

    https://github.com/dgl/haphash

    He might be willing to share the uri/paths to approve in your proxy config to deploy haphash or anubis etc.

  • permalink
  • fedilink
  • source
    •