Most DRMed ebook filetypes can be converted to non-DRM ones using Calibre and addons for it. That might cover comics, but I can't confirm as I've never been into digital comics.
Apple's iTunes used to let you pay like 30¢ more to get certain tracks in mp3. Others have mentioned bandcamp. About a decade ago Amazon used to be straight mp3 downloads for purchased tracks, but I'm not sure if that's standard anymore.