You could use matrix and host your own instance.

You could use Matrix but it is very finicky and complicated.

You could use XMPP but they don't have any nice clients.

You could use Zulip but it's confusing as fuck to navigate.

You could use Mattermost but it's tied to a corporation that seems intent on removing features from the open source version to convince you to buy a license.

You could use Quiet (not self-hosted but p2p) but it's very new and very Alpha.

You can use NextCloud Talk but that's probably more than you need, and it seems to be very difficult to maintain for many.

I won't engage in any arguments, these are just my opinions, and options for OP.

I've had a little snikket (prosody docker, xmpp) sever running for a year. monacles or cheogram are just fine, playing games with the fam in the chats is really cool.

dino or gajim on desktop.

calls work well, but android doesn't treat it as an actual phone call (home assistant automations based on phone state won't work)

and I've just now gotten direct messaging tied into home assistant, so i can use it for general notifications. it'd be awesome to get encryption though.

omemo is a bit of a pain, but they've all caught on.

but yeah i mean, very little hassle

I wish Rocket Chat got more attention

Yunohost comes with a builtin xmpp chat server.

https://yunohost.org/

There are many solutions. I have chosen xmpp/ejabberd/conversations/monal.

Irc with convos or the lounge as a web interface.

And yes, that's what I am running.

Why no-one has mentioned Delta Chat yet? https://runtimeterror.dev/self-hosted-chatmail-relay-for-delta-chat/

IRC, though you'll want to use it over TLS.

XMPP, which someone else listed, is also good if you want a more instant-message-like interface.

due to it missing ideal features

what features do you want? kindly elaborate

XMPP with Snikket could be an easy solution. If you don't want to talk to the wider web make sure to disable federation.

Matrix would probably be one of the better options, but xmpp is a pretty good choice as well.

Matrix, also PeerSuite if you don't wanna leave a paper trail as PeerSuite discards your sessions by default.

What is "ideal" for you? I use xmpp but some people want stickers and shit and use Matrix instead, it's much more heavy weight but also has these extra features. There's also rocket.chat that has all the extra stuff but is not built for federation.

I’m in a same boat, honestly.

Matrix has decent clients but managing a matrix instance is a world of pain, especially if you federate. And its resource use is really bad then: a single user instance can easily demand 4gb ram if you are in a couple popular chatrooms. Key propagation is oftentimes broken. Clients all have mixed support of features.

Xmpp is a joy to host, but there are no decent clients for iOS.

IRC is easy to host, but the IRCv3 coverage for clients is also meh.

I was looking for something that I could throw at casual people with relative ease and there's just not a thing. Even the "techy" chat is in discord nowadays.

I personally use Matrix for this. Artyom kinda summarizes issues of all chats pretty well. If you use Yunohost installing Matrix is at least easy. Matrix/Element is good enough and even my non tech wife is able to use it.

https://mattermost.com/

Is essentially a self hosted slack.

Maybe https://jami.net/ ?