Yeah so someone Aspie who is actually trying to be considerate will be lumped with those who are Aspie and inconsiderate.

I suppose it is like being a foeigner in your own country. Some foreigners may be arseholes but the ones who are not are still subject to disdain from some racist locals who jump on any chance to look down on non-compliance.