The latest version of that encryption adds features that are beyond what Signal does. It would be indeed nice to have in more clients, but this is not an disadvantage over Signal at all.
And there is a method to cross-sign keys that makes verification quite easy, but it comes with other trade-offs.
Of course if you have many servers there will be a variety of different ones. As I said there are trade-offs. Still better though than having only one single option that is under one of the worst possible jurisdictions.
Edit: you don't need to do that, there is TOFU for that, and that is entirely sufficient for larger groups where trust is necessarily limited anyways.