Old phone which is supported by (even an old version of) LineageOS, kept permanently in airplane mode.
Can be extra cheap in countries which have switched off 2G+3G networks, and old phones are more likely to have a headphone jack. You might even have one kicking around in a drawer.
Audio output quality and headphone driving capability varies -- research on audio forums for what might be good. LG v-series were very good.
If you're feeling adventurous you could crack it open and remove/short antennas & microphones and cameras.
If it's usb-only or the audio section is poor quality, apple sell a cheap usb-c dac that's pretty good. (get it from a reputable source -- most are counterfeit).