Interesting... Good to hear!

Thanks, it seems faster in general now too.

Thank you for the hard work.

Sorry, I am in the process of moving and have not been able to keep a close eye, and the disk filled up. I have increased the disk size now, but I see some errors with the images. I need to sleep now but in the morning if it is not fixed I'll figure out what is happening.

naw, just one or two

Sorry, I missed this! But I think that this is due to bots. From time to time, bots attempt to scrape the instance and this makes the site a lot less responsive.

I try to manually block these IPs, but sometimes new IP ranges show up.

So, the server that hosts the front-end via Tor will see the exit node connecting to it

The onion eliminates the use of exit nodes. But I know what you mean.

I appreciate the explanation. It sounds like replicating the backend and DB on the Tor node would help. Not sure how complex it would be to have the DBs synchronise during idle moments.

Perhaps a bit radical, but I wonder if it would be interesting to do a nightly DB export to JSON or CSV files that are reachable from the onion front end. Scrapers would prefer that over scraping, and it would be less intrusive on the website. Though I don’t know how tricky it would be to exclude non-public data from the dataset.

I have experienced issues both over tor and over clearnet. The tor front-end exists on its own server, but it connects to the mander server. So, the server that hosts the front-end via Tor will see the exit node connecting to it, and then the mander server gets the requests via that Tor server. Ultimately some bandwidth is used for both servers because the data travels from mander, to the tor front-end, and then to the exit node. There is also another server that hosts and serves the images.

What I see is not a bandwidth problem, though. It seems like the database queries are the bottleneck. There is a limited number of connections to the database, and some of the queries are complex and use a lot of CPU. It is the intense searching through the database what appears to throttle the website.

psy is on /this/ instance. Note my question was directed to the mander admins, who would only have the power to make me a mod on the node under their control.

If the Psy community is on a tankie instance, make a new community. Otherwise, request to be made moderator. That's my rule of thumb at least.

I’ve not tried the onion instance since reporting the data loss issue, but in principle a the onion host could be a good candidate for read-only access (scraping).

Would it perhaps make sense to redirect the greedy subnet to the onion instance? I wonder if it’s even possible. The privacyinternational website used to auto-detect requests from Tor exit nodes and automatically redirect to their onion site. In the case of mander, it would do that for the subnet giving problems. They are obviously not using Tor to visit your site, but they could have Tor installed. You would effectively be sending the msg “hey, plz do your scraping on the onion node,” which is gentler than blocking in case there is more legit traffic from the same subnet. That is assuming your problem is not scraping generally but just that they are hogging bandwidth that competes with most users. The Tor network has some built-in anti-DDoS logic now, supposedly, so they would naturally get bottlenecked IIUC.

I guess the next question is whether the onion site has a separate allocation of bandwidth. But even if it doesn’t, Tor has a natural bottleneck b/c traffic can only move as fast as the slowest of the 3 hops the circuit goes through.

Working great now, thank you!

This morning I woke up and a new IP sub-net (43.173.0.0/16) was excessively hitting the site from multiple IPs, probably scraping, making the site unresponsive. I blocked that sub-net and the site is responsive again.

Thank you.

A few days ago, I blocked several IP ranges to solve this. I unblocked them about two days ago in an attempt to solve some federation issues... The bots from this IP range came back.

This time I blocked only the IP range that has the most bot-like activity. Hopefully that resolves it.

Got a few minutes of 503504 gateway timeout errors here, but the pages just loaded back up again.

No problem! Happy to contribute

Ty for your efforts

Thanks!

I don't see those specific IPs, nor 16514. But now I see what scrapers tend to look like in the logs :)

I am now pretty sure that the cause was scraper-like activity coming from the Mlmym front-end that I am serving over an onion site. I am not sure if it randomly started mis-behaving or if a tor scraper was using it.

After blocking this, federation was restored, performance increased, and CPU use came down:

That's a level of sophistication I hadn't seen - using Tor to scrape. Hopefully they give up soon. I'm glad we got some respite. Thank you!

I just realized that it is not a 'scraper', the requests came from the server that I am using to provide an interface to the site as an Onion site. The amount of requests was suspiciously high so maybe a bot is scraping through Tor. I will leave it off for a few days and see if I can turn it back on later.

Thanks! It seems like the lag may have been due to a scraper

I did not update or change anything in the past few days.

But, now that you mentioned an AI scrapper I looked into the logs and noticed some heavy requests to the API from a specific IP.

Requests look consistent to scraping - just consistently and continuously issuing GET requests to different API endpoints.

XX.XX.XX.XX - - [14/Oct/2025:21:28:54 +0000] "GET /api/v3/community/list?limit=20&sort=TopAll HTTP/2.0" 403 107 "-" "Mlmym"

I have started denying their requests and it is the first thing that seems to have actually helped!

I don't want to speak too early but I think you may have identified the cause. Thanks!

Did you upgrade Lemmy on the 13th or something? Could be a new bug.

Also, did site traffic increase at all? Maybe there's an AI scraper messing up.

So far, I have been able to 'control' the CPU use by setting limits to the process that pulls stuff from the database (pool size, CPU, memory).

This does release some of the CPU for other tasks, but I think that that what creates the lag might actually be the clogged database queries. So, constraining those resources might not solve the lag problem.