But the things we do perceive are mostly accurate, with individual differences, and a hint of specializing bias (fruit colors). What, you think halucinating things that don’t exist isn’t detrimental to your survival (and mating chances)?

Yeah, Donald Hoffman ran thousands of evolutionary simulations in his cognitive lab. He created a bunch of creatures that perceived the simulated world accurately, and a bunch that just perceived fitness payoffs and used cognitive shortcuts to oversimplify their perceptions. Fitness always beats truth. Truth-perceiving organisms go extinct. Your ancestors were the early animals who perceived the world in fitness payoffs, not truth.

But there’s also the psychological part. And in some cases neurological, like born in the wrong body and such.

That's not biology, it's psychology. And it's not sex, it's gender. The sex-gender distinction is really important to trans people's wellbeing.