I guess I can see some ways that “lady” or “ladies” could sound sleazy but I’m surprised if this is someone’s first or only association. “Lady” is literally a landed title, the equivalent of “Lord,” and the highly dignified counterpart to “gentleman.”