This is genuinely the most useful reply I've gotten - thank you for taking the time. The breakdown of what the phone number actually does for is clarifying; I'd been thinking of it purely as a privacy leak, but you're right that it's also doing the "shared identity token" job, and dropping it means I have to solve that some other way (I use a locally generated ID, but I'll admit discovery/trust-on-first-contact is the weakest part).
The supply chain point is the one that lands hardest, and it's fair. "Trust me, the developer" is worth nothing if the code underneath isn't inspectable - I hadn't framed it that sharply to myself until you put it that way.
One thing I'd like your read on, since you clearly think about this properly: if a small project gets to open source + standard vetted libraries + a transparent build, but realistically can't afford a full third-party audit for a while - is that a "come back when you're audited" situation, or is open + standard libs + reproducible builds enough to be worth a careful person trying? Trying to understand the actual order of priorities, not just the wishlist.