the problem isn't listening to music on a speaker in public. The problem is that 99% of the people who do so put it on max volume all of the time.
If you're out in the middle of nowhere and there's no traffic noise to compete with, then 10% volume is probably louder than you need to hear it and also not bother others on the trail.