I spent decades feeling that way. I knew I was trans (or, at least, that I'd rather be a woman and low key despised my masculinity). I was convinced that I'd never be seen as a woman, that my frame was too big, my facial features and hair too masculine to ever pass.
Eventually, in my late 30s, I hit a point where I decided that I didn't care if no one ever wanted me again or if I never passed, I decided that I'd rather die an ugly woman than a man.
Joke's on me, I guess. ~4 years in and I could not be happier. It turns out that surgery and hormones work- people I knew well literally don't recognize me sometimes even after talking for a long time. It's not the most important thing, but I mostly pass in the world. Fwiw, I thought I was somewhat attractive pre-transition, but I'm fucking hot these days.
Regret is complicated and I love my life, so its hard for me to say that I regret waiting exactly, but I will probably always wonder What life would have been like if I'd started sooner.
Every time I swept those feelings under the rug, they came back bigger, more painful, and harder to deny. Only you can decide what/when is right for you, but I'd invite you to ask yourself "what if things went well?" and not just "what if I never pass?"
Idk, you'll figure it all out one way or the other, but if you're anything like me, those feelings only get bigger.