It's not a chance or a mere claim by "some people"; we already know what it was like before the secret ballot. This idea has already been tried. You'd date somebody who voted for a monster and they'd tell you "my boss/landlord made me do it".

This is where some people will so cleverly suggest "oh simply make it illegal to fire them for that cause"... which will work about as well as every other worker protection that isn't a union. They'll build a case to fire you "for another reason".