even though the server is open source, it isn't self hostable
Since its a centralized server that isn't self hostable, you have no idea whats running on their server. Signal even went a whole year once without publishing any server back end code updates, until it raised a lot of hackles so they started adding to it again.
But the signal foundation is a non profit with external audits and a proven track record with law enforced requesting data and getting basically nothing (If i remember correctly they only have your user to phone number relation and the last time you were online)
You have no idea what they give to authorities: in fact with NSL's, its illegal for them to tell you. Signal's response to this is "just trust us".