I find that for me this happens especially often in 3rd person camera control. And I think it has to do with the distance between the camera and the controlled actor.

Think driving a car, and the camera moving up close to the car when you're under something like a bridge which would otherwise clip the camera. At that point my preference for camera controll switches from 'orbit object' mode to 'aim' mode so to say.

Does that make sense?

I think the core of this dispute is the intuition of each person of what exactly they're moving and how they're moving it.

Like I have never once considered, until just now by reading your comment, that the stick could controlling my characters head. I've always intuitively thought that I'm controlling "the point in space that my character is looking at."

Changing this makes one way make more sense than the other. Like, despite my preference for non inversion, I would fully concede that if I were to imagine the stick controlling the head directly that inversion makes plenty of sense.