Congrats on your project! Sounds interesting. I’ve got a few things for you to consider.
The phone number is often (but not always) there as a shared identity token — you need to have something by which you can establish trust between the two endpoints, and most people have a phone number.
In cases like Signal, it’s hashed locally and sent to them as a hash value that can be “discovered” by other people who have never connected with you before.
In cases like Apple Messages, it’s used as an optional part of 2 factor authentication and as an SMS routing avenue — purely optional, as you can use an email address / Apple ID instead.
In many other cases, it’s used as a way for the messaging provider to track who you really are.
On to the “trust” bar — Open source and using industry standard libraries, at a start, with a real person and a physical address on the other side, and a fully transparent and auditable build chain.
These days, it’s really really common for a malicious actor to stick something into your supply chain that will give them full access to all the devices using your solution… at some point in the future during a series of innocent looking updates. If I don’t have full trust in all the upstream code being used by your messenger, I can’t trust your messenger either, no matter how much I trust you as the developer.