Lots of people in 1960s US were bothered by schools being integrated. Perhaps even the majority in many southern states. Does that mean we should have pandered to them?
Just because some idiots think it's a problem, doesn't mean that we need to listen to them.