Is it? You can have female hormone levels and still have male genitalia. Science is cool like that. And no, people aren't defined solely by their genitals, but they aren't solely defined by their appearance either, or their biology. It's a big mash up of those things and a few more.

Point being it's really hard to describe that situation as being solely one or the other. It's definitely something that needs to be indicated for medical reasons though, but most Identification Documents don't draw any distinction between sex and gender, much less allow for notation of something like that. Once again, paperwork lags behind reality.