Yeah, especially when one is likely much more powerful than the other. If you are a monk with a sword you are wasting your time. If you are a Warrior* with a free hand you are wasting your time.

*Sorry, that should have been Fighter, I'm sick, and I've been reading too many variant rulesets while I'm sitting at home.