That logic falls apart real fast. Hitting children, for example. Legal, but is it right?

Some places have legal laws that are horrifying. You can see the other implementation of your logic there. Like, is being gay a bad thing, as long as it is illegal?