"Are we the baddies?"

Technically, the Palantiri weren't the evil eye of the Sauron. Sauron and Saruman did use them (and that's why you see Sauron's eye in one of them), but the stones themselves were basically just communication and surveillance devices. They were, themselves, fairly neutral and not made by anyone especially evil (probably Feanor - flawed but not evil). But they were used for evil purposes during the time in the LOTR trilogy.

You think batman having a hidden lair as literally the tallest building around is unrealistic until you see what modern corporations are like

The palantiri themselves aren't inherently evil; Sauron was just a master of deception and used them as propaganda devices