Hot take maybe. If you don't like downvotes just use an instance that has them disabled. I'm not using Lemmy for actual discussion most of the time and neither are probably 99% of other users. I don't want to see the nazi adjacent comment and don't think debating with them is really worth anyones time but someone else can try.
Let's say you think people with Nazi adjacent views can be downvoted/banned. Where is that line drawn then. How nicely do they have to frame eugenics before you shouldn't downvote it because it's "thoughtful". Someone JAQing off in the comments about race statistics?
The line of where people do and don't downvote is not something that should be decided by instance admins/ community mods because it's one of the few things a user has total control over. I find it mildly alarming that vote are public but I understand that it's inherent to the system.