I am a 5'9" woman and prefer guys around my height, anecdotally all the women I know who want tall guys are themselves short.
It's a weak preference but yeah I think since I go through life being around the same height as most guys I meet I don't have such a strong association with height as a sex difference, if that makes sense.