I have a feeling there would still be a lot of ways for the rich to dodge the taxes, while regular people would somehow bear the brunt of it.

But, yeah, exempting anything below a couple million worth AND the primary residence might be a way out. Although, you'd probably need to define what a "residence" is, as well as put a cap on that, because otherwise those bastards would start building whole city-scale complexes and calling them their "primary residence".