And in literally zero of these cases was anyone's gender identity relevant to their crime. Raping someone is already a crime. Filming someone changing or toileting is already a crime. If a cis man goes and has a wee in the ladies' room and washes his hands and leaves, I don't care. If he's a member of the alt-right and he claims to be trans to get attention on social media, I don't care. If he rapes someone, then I care. If he films someone, then I care.
A lot of conservatives say the only way to protect people from a bad guy with a gun is a good guy with a gun. I propose an analogous principle: One way to protect women from a raping man in the loo is to have a good man in the loo with them. By segregating the toilets, you're ensuring that the only cis men in the ladies' room are either cleaners, plumbers, or rapists. Desegregate, and there will be lots of good men going to wee in the gender neutral washroom who can stop a rapist.
I make the above recommendation largely in jest, but the end result is something I believe in. A world where we all just go into the stall to have a wee, pull up our trousers, wash our hands, and leave. Your gender identity or what bits are inside your trousers does not have to be relevant, other than the principle that there should be a sanitary bin and a hygiene product dispenser.