Lemmy Release v0.19.20 - Reduced Memory Usage News & Update Threadiverse (join-lemmy.org)
submitted 3 months ago by to c/fediverse@piefed.social
 

What is Lemmy?

Lemmy is a self-hosted social link aggregation and discussion platform. It is completely free and open, and not controlled by any company. This means that there is no advertising, tracking, or secret algorithms. Content is organized into communities, so it is easy to subscribe to topics that you are interested in, and ignore others. Voting is used to bring the most interesting items to the top.

Changes

This release significantly reduces memory usage for the Lemmy backend. Metrics on production instances show a reduction up to 10 times. Here you can see the statistics from a few different instances. Read on for a technical explanation below. Note that the change only affects x86, there is no difference on ARM (e.g. Raspberry Pi), see here for details.

lemmy.ml (overall RAM usage including OS, Docker, PostgreSQL etc):

leminal.space (backend container only)

lemmy.world (backend API containers)

lemmy.world (backend federation and scheduled tasks)

How was such a major improvement possible? Memory usage in languages like Rust or C is managed by a so-called memory allocator. It requests large chunks of RAM from the operating system, and provides smaller chunks when needed in the program. For example, every string (such as post titles or markdown text) require chunks of memory to store them. When processing the data is completed, the memory should be freed and released, or reused.

Until version 0.19.19 Lemmy used the mimalloc memory allocator, which is supposedly better than the default glibc allocator. Yet a recent blog post points out that mimalloc does not play well with the tokio async runtime. It suggests to use jemalloc instead. Changing memory allocators is very simple in Rust, it only requires a single line of code. So we tried that, deployed the change on lemmy.ml and it immediately proved effective.

Upgrade instructions

There are no breaking changes with this release.

Follow the upgrade instructions for ansible or docker.

If you need help with the upgrade, you can ask in our support forum or on the Matrix Chat.

Thanks to everyone

We'd like to thank our many contributors and users of Lemmy for coding, translating, testing, and helping find and fix bugs. We're glad many people find it useful and enjoyable enough to contribute.

Support development

We (@dessalines and @nutomic) have been working full-time on Lemmy for over five years. This is largely thanks to support from NLnet foundation, as well as donations from individual users.

If you like using Lemmy, and want to make sure that we will always be available to work full time building it, consider donating to support its development. A recurring donation is the best way to ensure that open-source software like Lemmy can stay independent and alive, and helps us grow our little developer co-op to support more full-time developers.

no comments (yet)

sorted by: hot top controversial new old
there doesn't seem to be anything here
10 comments in c/div0@lemmy.dbzer0.com view that post
[โ€“] [S, M] 8 points 3 months ago* (9 children)

All done. Unfortunately it seems tesseract doesn't like this version. I'll see if there's a workaround

  • source
  • hideshow 9 child comments
  • [โ€“] 5 points 3 months ago (8 children)
  • [โ€“] [S] 7 points 3 months ago (3 children)

    Holy shit. I thought it was just an API version mismatch! The turbolib dev literally hardcoded our instance into a blacklist! Guess we have to decommission that frontend now.

  • source
  • parent
  • hideshow 3 child comments
  • [โ€“] 2 points 3 months ago (1 child)

    Well technically it was a downgrade from having our own special warning to a simple block, but yeah. I remembered reading that newer versions of Tesseract had flexible version support, so I was really surprised to see that message, had to check what was happening. I'm bummed tho, now I have to get used to a new UI :(

  • source
  • parent
  • hideshow 1 child comment
  • [โ€“] [S] 1 point 3 months ago (3 children)
  • [โ€“] 2 points 3 months ago (2 children)

    Interesting. I'm curious to know what their reasoning for that is

  • source
  • parent
  • hideshow 2 child comments
  • [โ€“] 2 points 3 months ago (1 child)

    The commit I linked added dbzer0 to the same blacklist, removing a custom warning that said

    "This instance and its admin staff encourages identity politics, groupthink, mob mentality, and extremist solutions to societal problems. Users who advocate violence are not moderated so long as the admins agree with the target. Caution and critical thinking are advised when interacting with this instance or its users."

    So probably something along those lines.

  • source
  • parent
  • hideshow 1 child comment
  • 7 comments in c/fediverse@lemmy.zip view that post
    [โ€“] 3 points 3 months ago (1 child)

    A miserable pile of secrets.

  • source
  • hideshow 1 child comment
  • [โ€“] [M] 1 point 3 months ago (4 children)

    Not sure why the proper title failed to fetch. Sorry about that.

  • source
  • hideshow 4 child comments
  • [โ€“] 1 point 3 months ago (3 children)
  • 1 comment in c/tecnologia@lemmy.eco.br view that post
    [โ€“] 4 points 3 months ago

    Maravilha. O uso de recursos do Lemmy sempre foi apontado como um problema. Parece que finalmente acharam uma soluรงรฃo eficaz. Sรณ falta conseguimos um bom alocador para programas que usam Tokio que suporte todas as arquiteturas.

  • source