But I recognize that such ideals haven't really succeeded in western society; so maybe it's easier to wear what you want as a particular traditional gender that you pick. And that's really just another way to look at the same thing. And maybe society understands that more?

I don't speak for everyone but I think for some people this is a big part of it. Fighting for what you just described here is hard enough already. Many people just want to live their lives and be happy. Also, non-binary people exist and are generally considered to be a part of the trans umbrella. I think the first step to building a society where gender truely doesn't matter starts with building a society where all trans people are welcomed and accepted.