Incomplete understanding can lead to wrong conclusions...

I stepped in "it" recently and I'm here to confess the errors in my understanding.

In photography I've run across for years and years loads of marketing promises, distortions, half truths, selecting data that fits preconceived curves, bending of reality, and outright lies.  

It's in part why I've written here and elsewhere about what I've found.  Methods, processes, setups are explained so that anyone can see for themselves what's what.  You can't (and shouldn't) blindly trust me.  Trust if you must, but verify.

Recently falling down a Wabbit Hole labeled "Fat Pixels", I came to a set of conclusions that, in retrospect, were malformed.  I used DxOMark's color depth and dynamic range numbers to evaluate the image quality of Sony A7, A7R, and A7S cameras.  Based on this I made decisions on gear purchases and spoke ill of the "Fat Pixel" discussion, particularly as it related to the Sony A7S and its "Fat Pixel" 8,3micron sensor site sizes.

What I was trying to evaluate was whether there might be some magic "Pixie Dust" in the small 12mpixel system that I might find good reason to enjoy in my own image making.  Of course I did nothing to evaluate Sony's own 15+EV dynamic range claims.  I assumed (bad, bad, bad, I know) that Sony was simply wrong and gone overboard in their marketing claim.

Well, the actual error I made came from my lack of understanding how DxOMark's numbers are calculated.  I relied on the numbers alone to tell the whole story. In my defense, when I read through DxOMark's comments I didn't see anything that led me to believe they were considering anything but the entire sensor output.

However, reading criticisms of DxOMark I've come to understand that sensor output is _downsized_ from whatever the native sensor resolution is down to 8mpixel before an evaluation of dynamic range and color depth are made.  Right here is the source of the error.

I should've guessed at this and wondered at the time how the numbers were coming out the way they were.  Looking at "test" results for various Sony Full Frame cameras we can see that the higher megapixel count sensors ALWAYS score better than the smaller sensors.  Always.  This is true for color depth and dynamic range.  Spend some time on their site to verify my claim.

How can this be?

In the process of downsizing an image for evaluation, noise is averaged out, dynamic range increases (by lowering the noise floor), and colors become purer.  I know this from my three image stack/average comparisons.  Noise averaging works similarly when downsizing.  We're not seeing anything that comes straight off the sensor when we read DxOMark's numbers.  The image has passed through some kind of conversion process.

There are other sites that compare and test sensors in camera systems, of course.  But these convert from the original RAW file into TIFF.  So we're not seeing anything straight off the sensor there, either.  While this might be a different process to the one DxOMark uses, there is a conversion of the image and I'm not convinced this won't effect the output.  There's no method evaluation information that I've easily come across that guides readers one way or another.

How, then, do we get at an understanding of color depth and dynamic range?  How do we get beyond the various comparison site's use of image conversions which can alter evaluations and results?  If someone knows, I'm all ears.  Please.  I'd like to learn.

As is often the case when I look into commercially available photography tools, I turn to Real World to see what's going on.  In this way, I found a site that allows people to download Sony A7S AWR files.  Grabbed a couple images and put them through my normal Rawtherapee processes.  This is something I should've done earlier, but am happy I finally got around to doing this now.

I can't quantify what I saw, of course.  There is nothing being measured.  All I can say is that A7S AWR files look like great starting points for still image processing.  They look very subtly different than my A7 24mpixel files.  The A7S colors are deep and rich.  The resolution is outstanding after being passed through Capture Sharpen.  The low ISO noise levels are very low, indeed.

Perhaps I should after all pick one up and try it?  I still have a few Nikkor lenses to sell.  Then we'll have a look.  A7S prices are like A7 around these parts.  Cheap, and getting cheaper by the day.