Lefty here. I have a queer wife who identifies as non-binary but still identifies as my wife, as well as plenty of trans and queer people in my social circles, and I don’t see the need to word police myself over completely innocent phrases. I don’t think you’re harming anyone by just saying women, when the men or nonbinary people who can get pregnant constitute like a tenth of a percentage of the population.
I go out of my way to make sure the not-straight people in my life feel safe and comfortable around me, but there’s a certain level of pearl clutching over language that I don’t feel the need to engage with. You aren’t being hateful, you’re treating people like human beings, and you get to decide for yourself how you speak.