Generally, the issue is that services want to blend the data into a single stream, otherwise you end up with having to make a separate connection to and from each client, each at full bandwidth called full mesh. It becomes easier to just have the encryption to the server, then decrypt all the streams from all the participating clients, and merge them into a single stream outbound. Adjust and re-encrypt that single stream and send it. That also allows for more control to make low bandwidth or dynamically changing bandwidth clients work better as you can adjust one stream rather than hundreds of outbound to each client. But that means the server has access to unencrypted streams to analyze and record. This is called server mixing.

Previous software generally only supported one or the other of those options. Signal is one of the few that have developed the technology to allow for selective forwarding which is the third option. Their website details how this works and it's open source so some others have adopted it or have come up with similar approaches.