Yeah, this really captures why I hate the mass rollout of AI on a philosophical level. Some people who try to defend AI argue that biases in the models are fine because these are biases that exist in reality (and thus, the training data), but that's exactly why this is a problem — if we want to work towards a world that's better than the one we have now, then we shouldn't be relying on tools that just reinforce and reify existing inequities in the world
Of course, the actual point of disagreement in these discussions is that the AI-defender often believes that the biases aren't a problem at all, because they tend to believe that there is some fundamental order to the world where everyone should just submit to their place within the system