In Anglo-Saxon, "man" meant "person," like Mensch in German still does.

"Woman" was "wyfman"-- female-human-person. And because sexism has ancient roots, the default gender assumption for a person was that they were male.

So it's not quite as straightforward as you propose, and the gender-neutrality you mention wasn't really all that wonderfully gender-neutral after all.

So how about this, based more on modern usage: "person" for a person, regardless of what their gender might be.

"Man" for those of male gender, however that gets socially assigned.

"Woman" for those of female gender, likewise.

And for all the edge and corner cases that make our world so richly diverse, well, we'll need to figure those out too, hopefully without being oppressive about it.