The last gen iPod nano touted a 30-hour battery life. Also, you don't need to peg the CPU for rendering audio - this can be accomplished with a very low-power DSP. The lack of radios also offers significant savings when compared to a smart watch, which you forgot needs to be able to receive notifications, not just wait for low-power sensor input.
I've professionally developed the firmware for several embedded systems and consumer electronics devices with very strict power requirements.