XMPP biggest issues:

• The absurdly and laugably slowness of the protocol evolution on critically needed features: it was bleeding edge years back. The XSF totally screwed up the VoIP extension management. Google dropped a working out-of-the-box solution on the table in 2005: "Take it and use it! You're welcome!". They took 4 years to improve it before declaring the extension stable. Only then did clients start implementations, and bummer: the spec was not perfect, there were a lot of hiccups and "client A can make calls to B but not C" issues, because, who would have guessed, the only way to improve a spec at some point is to test it out there, and not stare at it and make some minor changes. By the time, most XMPP users of the time who wanted VoIP had moved on to others tools, XMPP went from at the top to very late. They're doing it again with MIX (next gen rooms): the first draft is from 2015. It's still "experimental", though some servers team started implementation (such lack of patience...)
• An ocean of servers/clients with no consistency. You get to Matrix, and you have Element for all platforms. It can be native code everywhere, but it's the same look and feel. You can tell your pops "just install Element". In the XMPP world, there is no equivalent, though Snikket seems to be going there (consistent UI across platforms): all clients are different, different UI, platform specific with different set of features, some are a 1 guy project. If you're not guided by a tek-savvy person, you have no idea where to start.

With all that said, I run a server for our family and its resources consumption is barely a blip on the radar. The lightest Matrix servers are an order of magnitude hungrier. And the difference increases with scale!

XMPP is the absolute best solution to multiply small servers, a very good thing for a healthy federation.