Haven't given too much thought to be fair. Taking video on the phone is much more obvious, while someone with camera glasses could make the excuse "I'm not recording!" and you'd be hard-pressed to prove it. For surveillance cameras, you could know where they are and evade, throw a rock, or drape something over them, whereas you'd have to go up and snatch the glasses off the wearer.
I also wouldn't be against it if it were used legitimately to help with a disability, or for specific tasks like a HUD with vitals, etc when doing surgery. But for general use, I'm not comfortable.