Eh, that's pigeon-holing it. There are many, many more reasons TERFs can hate trans women. Sometimes it's as simple as them being confused for "men" by bigots because they're not traditionally attractive and that pisses them off at their core, or sometimes they're standard religious bigots that think, "God made you a MAN so be a MAN!", ignoring all the evidence that even genetic sex is a spectrum, let alone gender.

Never underestimate the creativity of bigots to justify their hate by any means necessary.