Windows file manager opens things in apps based on file extensions, and then it's up to the apps to figure out what to do with it. I did a bit of testing, and it seems like Firefox is fine with opening JPEGs mislabeled as PNGs, but not PDFs mislabeled as PNGs. LibreOffice Draw is fine with that though, so if in windows I set that as my default for PNG files, it opens a PDF labelled as a PNG perfectly normally (and can also open actual PNGs normally).
If I just completely delete the extension from a PNG or PDF, Firefox will open either correctly.