I would argue, figuratively, not being tall as a man, may have the same effect as not being conventionally attractive has as a woman.

I may be queer where the culture is different, but I'm 5'5".

I'm aware of the ways that men can still harshly graded for their appearance. I think my statement stands. And frankly, to me men being expected to be tall feels like an extension of the idea their value is what they can provide.

There are other parts to the way men are related to or valued. But compulsory performance of attractiveness is not the same for men as it is for women. I have a messy relationship with gender and have walked on both sides of the fence and know a lot of trans folks who have also.

People are cruel to women that fail to adequately perform attractiveness. Even ones they're not in any way involved with or connected to.