Sauron wore it in the great battle, and he was by all accounts visible, otherwise how could Isildor cut the finger off? That would be a very lucky swing if Sauron was invisible.
No, I think its power depends on the wearer. Frodo didn't want to be seen, so it made him invisible to everyone but Sauron, who understands its power.