I shared a link from NPR Tiny Desk but there was no image preview or video embed. After investigating this I learned that when you do that, each instance picks up the link and individually retrieves the link preview nearly simultaneously. For gargantuan sites like YT or Reddit, this is a trivial amount of traffic and not a problem, but for smaller sites like NPR this results in a sort of botnet DDOS attack on the site's servers. More here. Another example here . For that reason, NPR has blocked Fediverse servers like Mastodon.

I have experienced this on my personal server as well (DDOS), but at the time didn't know why, exactly.

This isn't the end of the world, but certainly is not great and hurts engagement with the content. Is there a solution for this? Perhaps using some sort of webtorrent service like PeerTube where they just fetch the information from another fediverse server instead of the source server? Maybe fetching embeds client-side? At least that would stagger the requests.

E: Many people have replied comparing this to the Reddit "hug of death" or "The Slashdot effect". This is not the same at all. Those are legitimate site visits from humans that happen over the course of several hours. This is the result of several thousand servers all automatically fetching the same image simultaneously , and without any human involvement, and potentially next to zero human traffic to the site, which is why I compared it to a botnet or DDOS attack.

top 100 comments

sorted by: hot top controversial new old
[–] 215 points 3 weeks ago (25 children)

There are two proposed solutions I have seen. I do not claim that this is exhaustive, just only limited to the proposals that I have stumbled across.

FASP - Fediverse Auxiliary Service Provider

The idea that mastodon has is to have a third-party service that does something like generate a link preview once and then share that to fediverse instances that need it. They have a whole spec they wrote up about these kinds of services that can be run to help out fediverse instances with different things like this, search, or discoverability. They call these FASPs - the repo with the spec is here.

I don't yet know of any FASP that is live and running, so it is kind of a paper spec at the moment. I don't do too much on the microblogging side of the fediverse though, so I very well could be wrong.

Federate link metadata and the post at the same time

This comes from an issue on PieFed's repo. It is basically just an idea at this point, but it is a pretty basic one; have the originating instance fetch the link preview information and simply federate it out alongside the post itself.

  • source
  • hideshow 25 child comments
  • [–] 126 points 3 weeks ago (12 children)

    That second solution is what I thought of as doing as I read this. It doesn’t rely on a third party service being up. And that instance has to do that work anyway. So why not just share the results with others?

  • source
  • parent
  • hideshow 12 child comments
  • [–] 22 points 3 weeks ago (5 children)

    how would you know the results havent been tampered with?

  • source
  • parent
  • hideshow 5 child comments
  • [–] 117 points 3 weeks ago (2 children)

    The same way you know the rest of the data hasn’t been tampered with - you don’t. And when you find untrustworthy instances, you defederate from them.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 15 points 3 weeks ago (1 child)

    That's not true, you know the data hasn't been tampered with because of HTTP signatures. The fediverse isn't just "yolo, just trust me bro". There are some parts that are a bit more towards that side (particularly with groups on Lemmy) but I don't think this should be.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 9 points 3 weeks ago*

    HTTP signatures are not a default feature, the baseline is ECDHE key exchange with forward secrecy and deniability. Headers are not typically signed.

    There isn't even an official standard for redistributable signed page metadata. There's just prototypes. I absolutely hope one of those prototypes becomes official, but we're not there yet.

    The activitypub fediverse is literally very extremely Yolo trust me bro, it works because you can see the originating server with everything and misbehaving ones gets banned.

  • source
  • parent
  • [–] 10 points 3 weeks ago (5 children)

    It was my first thought too but there are downsides to it. Trust is a small one given the nature of the system but compute and bandwidth costs could be an issue also, especially for larger instances.

    The first solution is more fragile but could be CDN'd and those are cheap.

  • source
  • parent
  • hideshow 5 child comments
  • [–] 6 points 3 weeks ago

    I don't think compute resources should be an issue, if anything it saves other instances resources.

    As far as I understand, right now, all instances are fetching this data individually, so they all have the same compute load from this. Doing it on one and sending it along with federation should keep resource useage pretty much the same on the originating instance, and less for all others.

    This would still increase bandwidth of course, but at least with all VPS providers I have used, bandwith is pretty cheap.

  • source
  • parent
  • [–] 5 points 3 weeks ago (3 children)

    WebTorrent kind of solution like PeerTube is the best, because it's highly scalable yet not that resource-intensive, because everyone (even users) provides their bit; also, as it's entirely P2P, it acts sort like a distributed and decentralized CDN, so it's snappier for everyone because it's more likely that you have a peer near to you like that.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 4 points 3 weeks ago (2 children)

    Absolutely LOVE this idea but I feel like a solid implementation would be a pain in the fanny (and it also requires other peers to be online). I dip in and out over the day myself but I don't know what others use case is.

    Also it is far less trustworthy and harder to block bad actors (V's removing a known bad actor from a relatively small peer group).

  • source
  • parent
  • hideshow 2 child comments
  • Idk, the part of uptime of peers is solved precisely by the fact that not every peer will be connected at the same moments that every other peer. If I'm connected at 10pm, it's highly likely that there will be at least another single peer at that hour that has what I'm pulling.

    As for bad actors, that's something that at the end of the day is unavoidable. Though with some automated systems could be possible to make it more hard for them, but it's not that different from a centralised approach, and thus is better because is computationally more efficient.

  • source
  • parent
  • [–] 69 points 3 weeks ago* (6 children)

    I'm a fan of the second one. My thought is, if you trust a server enough to accept messages from it, then you should also trust it enough to let it scrape the page title, article image, etc. for you, too.

  • source
  • parent
  • hideshow 6 child comments
  • [+] 1 point 3 weeks ago (4 children)
  • [–] 5 points 3 weeks ago (1 child)

    There's no reason to believe that this mechanism doesn't check for CSAM (I am also pretty stumped that this is actually a problem with Mastodon according to your comment?) . Lemmy had a phase where messed up stuff got federated, which led to @db0@lemmy.dbzer0.com's implementation of an anti-CSAM tool: https://github.com/db0/fedi-safety, which is based on the CSAM detection tool for the AI Horde: https://dbzer0.com/blog/ai-powered-anti-csam-filter-for-stable-diffusion/ - it's probably not so hard to adapt for Mastodon too.

  • source
  • parent
  • hideshow 1 child comment
  • load more comments (1 reply)
  • [–] 5 points 3 weeks ago* (1 child)

    This is no different from how it is already, though. If a user posts a CSAM link, your server immediately visits that link to scrape it, and you suddenly have that content on your server.

    Of course, if you have a blocklist of forbidden sites, then your server will know not to scrape that site. And the same thing applies if we allow servers to send pre-scraped metadata. If the incoming message is for a blocked URL, just drop the message and don't save any part of it.

    The only new threat here is for a malicious server operator to modify outgoing messages to include CSAM metadata on a harmless link. However, that possibility already exists with any other content posted on the fediverse. If a malicious server is modifying its own outgoing messages, the only fix is to defederate from it.

    Allowing servers to forward pre-scraped metadata to other servers does not introduce any new threats that do not already exist.

  • source
  • parent
  • hideshow 1 child comment
  • load more comments (1 reply)
  • [–] 1 point 3 weeks ago (3 children)

    But what happens when a link target changes? For example, I've seen news articles change as the author gets new information. Might be an edge case, but a solution like this would always show the preview at the time of the initial post.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 82 points 3 weeks ago (1 child)

    Thanks for bringing this up. I think in general we need to work on optimisation of Activitypub and Threadiverse implementations of it.

  • source
  • hideshow 1 child comment
  • [–] 11 points 3 weeks ago

    Either publishing server should serve the metadata - although it would be prone to bad actors, so a refetch in the next 0-180 seconds would be appropriate. Or just add a random 0-60 second delay before fetching the data.

  • source
  • parent
  • [–] 44 points 3 weeks ago (6 children)

    I have my blog address (which goes to my home server) in my Mastodon profile, and every time I post on Masto, my server gets a request from every Mastodon site that federates with my instance - i.e. hundreds of requests. It's no problem for my server, because my blog is a static site, but for anything that dynamically generates every single page impression (like worpress or joomla), it could be a problem. Caching settings might help, I suppose.

    Truth is, though, I get far more traffic from vulnerability scanners and AI crawlers that don't honour robots.txt (or aren't yet listed).

  • source
  • hideshow 6 child comments
  • [–] 12 points 3 weeks ago (1 child)

    that don’t honour robots.txt (or aren’t yet listed).

    Throw'em in the tar pit.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 3 points 3 weeks ago

    Limited mental capacity these days, so I don't have the spoons to properly research and implement a solution at the moment. I installed Nepenthes last year, but I've never managed to get round to integrating it into my site. It's low priority at the moment.

  • source
  • parent
  • [–] 2 points 3 weeks ago (1 child)

    To be honest, if a website has no caching of their server-rendered parts, they are also doing something wrong! Very little sites really need to re-render a public page on every view.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 5 points 3 weeks ago

    Around 2010, I used to work at an MSP, and once had a customer that ran a custom LAMP stack website for local radio UK stations web presence. It was a bit slow, running on a dedicated physical server, but every time it snowed, the website would 500 out - because of too many requests to the "School Closures" page. The site had no caching other than Apache's internal cache, but PHP pages could not be cached because "reasons", and the original developer was "not available". I optimised their MySQL indexes, increased the number of Apache workers, and it improved the render time per page and number of concurrent connections. Stress testing looked good.

    Anyway, some months later, 2AM one snowy morning, I was on call, and it went down again. This was when I discovered the school closures problem. After diagnosing, and then liaising with the customer to discuss the options, I was told "just make it work". I ended up modifying the PHP script to add some "on spec" caching that saved the HTML to a temp file in ramdisk, and only re-ran the query-and-render code if the cached copy was more than 5 minutes old. It was hacky, but it did the trick, and I was finished by about 4 AM. This hack was later applied at their request to a number of trouble-points in the source tree, as and when things broke.

    We recommended adding a Varnish cache, and found a more generalised caching library but they wouldn't/couldn't authorise engineering work for this - they were running on a shoestring - and they already had the tech break-fix contract with us, and decided it was sufficient to just spot fix things as they broke rather than fix the root cause.

    I left that MSP in 2012, so I don't know what happened after that, but as far as I know, the site could still be running.... unlikely, but possible.

  • source
  • parent
  • [–] 38 points 3 weeks ago (7 children)

    It's almost like the software that powers the internet was designed for a different time and is only still used because there's monied interest in keeping it around. We need GNUnet.

  • source
  • hideshow 7 child comments
  • [–] 15 points 3 weeks ago (2 children)

    GNUnet helps building a new Internet

    GNUnet is a network protocol stack for building secure, distributed, and privacy-preserving applications.

    With strong roots in academic research, our goal is to replace the old insecure Internet protocol stack.

    GNUnet provides privacy-by design

    It provides [improving addressing, routing, naming and content distribution](https://www.w3.org/2014/strint/papers/65.pdf) in a technically robust manner - as opposed to ad-hoc designs in place today.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 27 points 3 weeks ago (1 child)

    So we need people to just mass post links to bad websites, is what you're saying.

  • source
  • hideshow 1 child comment
  • [–] 42 points 3 weeks ago (14 children)

    I mean, just how many instances are gonna ping the URL? Several hundred, low thousands at worst?

    Unless it happens to be some very large media content or a dynamic page with no caching, I'd think the automated federation hits shouldn't be too bad.

    The influx of users that might come after a viral post though, that's rough....

  • source
  • hideshow 14 child comments
  • [–] 12 points 3 weeks ago

    Yeah but we can't say we want to encourage people to have/build/share independent smaller websites then beat them up when they are linked. That's a huge mismatch of morality. Either we want to encourage people off the big platforms and should make it easier for the little guys or we are blowing magic smoke.

  • source
  • parent
  • [–] 21 points 3 weeks ago (10 children)

    It's just bad design. What happens when/if the fediverse scales up to many more instances?

  • source
  • parent
  • hideshow 10 child comments
  • [–] 7 points 3 weeks ago (8 children)

    I liked writing efficient code but every place I worked didn't care at all about it. All they wanted was a finished feature and a kanban note moved to finished.

  • source
  • parent
  • hideshow 8 child comments
  • [–] 0 points 3 weeks ago (6 children)

    The many pointless presentations I've given about tech debt and code quality. Argued for redesigning poorly designed systems. They don't care. Just churn out more features to cross off the list, and why do those features take so long?

  • source
  • parent
  • hideshow 6 child comments
  • [–] 0 points 3 weeks ago* (5 children)

    Yeah I think a lot of us share this experience. All the stuff we are tought in our educations about clean code and architecture... Once we come out and work, management don't want that stuff if it takes extra time. They want features shipped as soon as possible, and every meeting is about "how fast can you do this feature".

    And now of course embracing AI since it's so fast. When things break, engineers will have to figure out the problem when the entire system is down, not management. :)

    And then they can have a post mortem where they say that engineering must have better routines...lol.

    I no longer work in tech but I quit at the top I think, before Ai was a requirement. And when consulting salaries was still good. Now they are awful.

  • source
  • parent
  • hideshow 5 child comments
  • [–] 1 point 3 weeks ago (4 children)

    No, you just worked at bad places

  • source
  • parent
  • hideshow 4 child comments
  • [–] 2 points 3 weeks ago (3 children)

    Sure, but there are a lot of bad places. I bet the majority of engineers work in bad places like this.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 1 point 3 weeks ago (1 child)

    I think so. I've worked at bad, average and good places, but the bad places tend to be larger and far less efficient, so they have to hire more engineers to make up for the inefficiency.

    And they don't necessarily pay well either. They see engineers as powerless cogs who have to do what they're told, so they get paid like that too. And then they complain they can't find seniors.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 3 weeks ago

    Pretty much. The engineering teams ive worked in had maybe one actually good guy, and the rest were awful engineers. I'm not saying I'm super good but I grew up with Linux and networks at home, and was programming from a young age. So im pretty decent. But most engineers had no experience with computers and was in it for the money, obviously.

  • source
  • parent
  • [–] 19 points 3 weeks ago* (2 children)

    Randomly stagger the times of each instance's request.

    +1-300 seconds

  • source
  • hideshow 2 child comments
  • [–] 19 points 3 weeks ago (2 children)

    they could include the link preview data as part of the federation

  • source
  • hideshow 2 child comments
  • [–] 17 points 3 weeks ago (3 children)

    Sounds like every instance is doing the preview generation. Maybe it could grab the preview data it needs on the original instance, and pass that around with the post...

  • source
  • hideshow 3 child comments
  • [–] 2 points 2 weeks ago

    Another option could be to have the originating Mastodon server provide the preview image in content-adressable way, and have a standard allowing websites to provide the same: if the hash of the file(s) provided by the first / a Mastodon server to another matches the one from the target website, no need to download from the target website. Only if they differ then you'd actual download the data from the target website.

    Each Mastodon server from the federation could quickly decide whether to trust that preview data from the peer it gets the rest of the post from or not.

    This is basically taking ideas from IPFS but applying it to just the preview data that ends up being downloaded over and over again. You could also make that part a blob of data stored directly in IPFS. But IPFS was pretty slow to resolve content, when I last tested.

  • source
  • parent
  • [–] 40 points 3 weeks ago (4 children)

    Is this the same thing as the “Reddit hug of death”?

  • source
  • hideshow 4 child comments
  • [–] 14 points 3 weeks ago (1 child)

    it seems like ideally it shouldn't attempt to preview the link before a user tries to view the content. i guess that would increase latency for users though

  • source
  • hideshow 1 child comment
  • [–] 11 points 3 weeks ago (1 child)

    Is this why the automated title retrieval never works for NPR?

  • source
  • hideshow 1 child comment
  • [–] 9 points 3 weeks ago* (last edited 3 weeks ago) (9 children)

    I don't really get it. The image is federated, there's no need for instances to refetch it. Just checked out some posts and indeed it's there, in the image property. With the URL pointing to the Lemmy instance.

    So if this is actually an issue, that's just bad programming and needs to be fixed.

    Edit: didn't notice OP uses Piefed. Unlike Lemmy and Mbin, it seems like Piefed doesn't federate it.

    Edit: actually I'm blind, Piefed does federate it too, sorry piefed

  • source
  • hideshow 9 child comments
  • [–] [S] 0 points 3 weeks ago (8 children)
  • [–] 7 points 3 weeks ago* (last edited 3 weeks ago) (1 child)

    I just took this random post from my frontpage: https://sh.itjust.works/post/68605733 and made a manual request for the federated JSON-LD.

    {
      "@context": [
        "https://join-lemmy.org/context.json",
        "https://www.w3.org/ns/activitystreams"
      ],
      "type": "Page",
      "id": "https://sh.itjust.works/post/68605733",
      "attributedTo": "https://sh.itjust.works/u/derrickoswald",
      "to": [
        "https://lemmy.world/c/technology",
        "https://www.w3.org/ns/activitystreams#Public"
      ],
      "name": "Stratospheric airship flew from US to Japan and back",
      "cc": [],
      "content": "<p>It seems with an unmanned airship they should be using hydrogen for the ~8% increased payload capacity.</p>\n",
      "mediaType": "text/html",
      "source": {
        "content": "It seems with an unmanned airship they should be using hydrogen for the ~8% increased payload capacity.",
        "mediaType": "text/markdown"
      },
      "attachment": [
        {
          "href": "https://arstechnica.com/gadgets/2026/10/trans-pacific-journey-tested-stratospheric-cell-tower-concept-in-japan/",
          "mediaType": "text/html; charset=utf-8",
          "type": "Link"
        }
      ],
      "image": {
        "type": "Image",
        "url": "https://sh.itjust.works/pictrs/image/df608ab3-b644-489d-b217-9224aa09e203.jpeg"
      },
      "sensitive": false,
      "published": "2026-10-19T20:16:37.383385Z",
      "language": {
        "identifier": "en",
        "name": "English"
      },
      "audience": "https://lemmy.world/c/technology",
      "tag": [
        {
          "href": "https://sh.itjust.works/post/68605733",
          "name": "#technology",
          "type": "Hashtag"
        }
      ]
    }
    

    Notice the

    "image": {
        "type": "Image",
        "url": "https://sh.itjust.works/pictrs/image/df608ab3-b644-489d-b217-9224aa09e203.jpeg"
      }
    

    Yes the image itself isn't sent over (iirc they never are?), but it's stored on sh.itjust.works. Other instances grab it from there, not the website the post links to.

    I did notice just now though that some Lemmy instances seem to just proxy the image through their servers, which I assume is to avoid CORS. But that's an instance choice, not a general fediverse issue.

    Mbin does the same as Lemmy, but it also includes the image in the attachment property. Probably for compatibility with some other platform.

    One thing I didn't notice before was that you're using Piefed. That's the one software I didn't check before writing my previous comment. It does seem like Piefed is the outlier here. I tried a post from piefed.ca and the image is nowhere to be seen. The image property is not present, it's not in the attachment array, and I can't see any other property that contains it.

    Edit: A piefed.social post I just tried did have an image, so I assume I'm just blind.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 3 points 3 weeks ago (2 children)

    Are you sure? In your post you're talking about fetching an image for a link preview, not an image post.

    Usually, when I see an image post, the source is actually the pictrs of the source instance, so in that case I think it is federated.

  • source
  • parent
  • hideshow 2 child comments
  • [–] [S] -1 points 3 weeks ago (1 child)

    No one was talking about an image post...

  • source
  • parent
  • hideshow 1 child comment
  • [–] 3 points 3 weeks ago*

    I figured a post with a link with preview image wouldn't be the same in terms of event as an image post. Turns out it pretty much is, but also, I checked a few posts and cross-posts across instances, all of them have an image url of the original instance, not the one I'm viewing on. So if they are individually fetching this image, they're not using it.

    Edit: I was wrong, two of the crossposts just didn't respond to the request for activity, so I was looking at the wrong response data

  • source
  • parent
  • [–] 2 points 3 weeks ago (2 children)

    the image should be federated

  • source
  • parent
  • hideshow 2 child comments
  • [–] 8 points 3 weeks ago

    Also https:// itsfoss.com/news/mastodon-link-problem/

  • source
  • [–] 5 points 3 weeks ago (4 children)

    Same thing used to happen with reddit giving sites the hug of death lol

    It's unfortunate but kind of expected.

  • source
  • hideshow 4 child comments
  • [–] [S] 10 points 3 weeks ago (3 children)

    Its not at all the same thing

  • source
  • parent
  • hideshow 3 child comments
  • [–] 0 points 2 weeks ago (2 children)

    You're right, it's not like... thousands of network connections are happening to the same site in both cases resulting in resource exhaustion.

    Oh, wait...

  • source
  • parent
  • hideshow 2 child comments
  • [–] [S] 0 points 2 weeks ago (1 child)

    Having things in common does not make them the same thing. I explained this in the OP.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 0 points 2 weeks ago

    You're right, neither of them are known for providing thousands and thousands of distributed network connections to the same destination resulting in that same destination server running out of resources. They're clearly not the same thing at all, no, definitely not at all.

  • source
  • parent
  • [–] 24 points 3 weeks ago (6 children)
  • [–] [S] 5 points 3 weeks ago* (last edited 3 weeks ago) (3 children)

    For gargantuan sites like YT or Reddit

  • source
  • parent
  • hideshow 3 child comments
  • load more comments
    view more: next ›