Yep.
It really becomes important not to value the opinions of others when you are different and outcast. Selectively, if you can manage that, but universally if you can't.
It helps that these folks operate on a lower standard of behaviour in how they treat others. It helps to understand they do these things to push down their own insecurities and to feed their ego. You can decide not to value the opinions of those who aren't worthy of judging you and you shouldn't try to appease them or put your energy into trying to control how they behave.
Put energy into learning how to properly not care.