You're right.

But, damn near anything that's said with mean intent about a visible minority long enough becomes a slur. So it is something of a game of whack-a-mole that continuously removes words from the language as collateral damage.

I personally think people should just stop being mean entirely, slurs or not.