Good read. Funny how I always thought the sensor read rgb, instead of simple light levels in a filter pattern.
post
This is why I don’t say I need to edit my photos, but instead I need to process them. Editing is clearly understood by the layperson as Photoshop and while they don’t understand processing necessarily, many people still understand taking photos to a store and getting them processed from the film to a photo they can give someone.
This write-up is really, really good. I think about these concepts whenever people discuss astrophotography or other computation-heavy photography as being fake software generated images, when the reality of translating the sensor data with a graphical representation for the human eye (and all the quirks of human vision, especially around brightness and color) needs conscious decisions on how those charges or voltages on a sensor should be translated into a pixel on digital file.
Same, especially because I'm a frequent sky-looker but have to prepare any ride-along that all we're going to see by eye is pale fuzzy blobs. All my camera is going to show you tonight is pale sprindly clouds. I think it's neat as hell I can use some $150 binoculars to find interstellar objects, but many people are bored by the lack of Hubble-quality sights on tap. Like... Yes, and then sent a telescope to space in order to get those images.
That being said, I once had the opportunity to see the Orion nebula through a ~30" reflector at an Observatory, and damn. I got to eyeball about what my camera can do in a single frame with perfect tracking and settings.
How do you get a sensor data image from a camera?
RAW files. Even then, you mostly see the processed result based on whatever processing your raw image viewer/editor does. But if you know how to get to it and use it, the same raw sensor capture data is there
Most cameras will let you export raw files, and a lot of phones do as well(although the phone ones aren't great since they usually do a lot of processing on it before giving you the normal picture)
My understanding is that really raw phone data also have a lot of lens distortion, and proprietary code written by the camera brand has specific algorithms to undo that effect. And this is the part that phone tinkerers complain is not open source (well, it does lots of other things to the camera too).
Modern mirrorless cameras do this too. For example, this is what my Canon kit lens looks like with/without digital barrel distortion correction:


Not my photos. From: https://dustinabbott.net/2024/05/canon-rf-s-18-45mm-f4-5-6-3-is-stm-review/
And https://dustinabbott.net/2024/04/canon-rf-24-50mm-f4-5-6-3-is-stm-review/

My own unprocessed RAWs are pretty wild. But (IMO) it’s a reasonable compromise to make lenses cheaper and better, outside of some ridiculous examples like the 24-50.
Not sure how worth mentioning it is considering how good the overall write up is.
Even though the human visual system has a non-linear perception of luminance. The camera data needs to be adjusted because the display has a non-linear response. The camera data is adjusted to make sure it appears linear to the displays face. So it’s display non-uniform response that is being corrected, not the human visual system.
There is a bunch more that can be done and described.
I'm a little confused on how the demosaicing step produced a green-tinted photo. I understand that there are 2x green pixels, but does the naive demosaic process just show the averaged sensor data which would intrinsically have "too much" green, or was there an error with the demosaicing?
Yes, given the comment about averaging with the neighbours green will be overrepresented in the average. An additional (smaller) factor is that the colour filters aren't perfect, and green in particular often has some signficant sensitivity to wavelengths that the red and blue colour filters are meant to pick up.
edit: One other factor I forgot, green photosites are often more sensitive than the red and blue photosites.
I think the penultimate photo looks better than the final one that has the luminance and stuff balanced, but maybe that's just me.
It’s not just you.
Zooming in, I feel like the “camera jpeg” lost sharpness to recompression.
It’s kind of insane that cameras either dump raw data, or do all this magic only to throw so much away to an ancient image codec that loses even more when recompressed.
Newer ones can save a HEIF or a “lossy RAW” in some circumstances (which is an infinite improvement), but still; I eagerly await the day cameras can save a JPEG-XL all by themself, and that I can post them on the Fediverse.
Added https://maurycyz.com/index.xml to my feed reader!
Is this you? If so, my wife wonders what camera and software you used!
Everything in the article notwithstanding (and it is true), even the notion that any image getting posted anywhere being "unprocessed" is hugely spurious even before you get into digital. As anyone who as ever developed film knows, during the development process you also have a huge amount of influence on how the final print will turn out.
In fact, come to think of it, that's probably why it's called a "process."
Anyway, Pierre over there with his beret and turtleneck and his designer spectacles thinks doing no postprocessing on his images makes him "more purist," or whatever the fuck.
I post unpostprocessed images (with several rather glaringly obvious exceptions) because I enjoy getting the initial output from the camera as close to what I wanted to begin with because that allows me to be lazy.
We are not the same.
One of the coolest projects I had during undergrad was in photography class. Digital cameras were not allowed. The teacher gave us an assignment to "photoshop" a print using only what we find in the red room (meaning no computers) to make a print that should be impossible. The final prints were unrecognizable from the initial subject matter. It was pretty amazing the amount of manipulation possible during development.
Several pieces of terminology still used in digital image manipulation today have names that are hangovers from the analog film era. The entire notion of "digital negatives," i.e. Adobe's DNG format is one of them. Also the doge and burn tools in your photo editing software, as I'm sure you know.
My camera does focus stacking, dynamic range stacking, high resolution shooting and ND filtering all within the camera. Yet they're all photos "straight out of the camera".
It really does highlight just how silly the idea of a "pure" photo is. It's all processed, and it's all processed with certain assumptions and preferences in place that other people might not share.
There’s nothing that happens when you adjust the contrast or white balance in editing software that the camera hasn’t done under the hood. The edited image isn’t “faker” then the original: they are different renditions of the same data.
You can still carefully choose which renditions you want to put out to the public to manipulate people's perceptions. The video of the uber self-driving car hitting a person being an example that comes to mind, where the video released by uber looks far darker than how human would typically perceive the same street while driving, which led to a lot of "even a human driver would have hit them" apologia. So even if "fake' isn't technically accurate, its still lying to the naive viewer.
Regardless, cool breakdown of what does into making digital photography work at a basic level.
Cool. Reminds me a little bit of literary interpretation. There's no such thing as "just reading what's there", everyone interprets.
This is part of why it’s so silly that people use the RGB values of pixels in an image to argue about the colour of real life objects.
I really enjoyed reading this. I kinda guessed this is what went on behind the scenes but it's nice to have everything layed out step by step.
all 35 comments