One last Wabbit Hole ~ and it's a whopping BIG one, too!

A little "accidental Renaissance"
to keep things interesting in prep for
reading through somewhat dense material.

 

[References section updated twice ~ 4 March, and 3 March, 2023]

After whinging and whining about the lack of verifiable, accurate knowledge, I find myself at the bottom of an Enormously Vasty Wabbit Hole.  At the top and just as I fell in I saw it was labeled  "Knowledge You Are Looking For."

The areas I wanted to learn more about but was having a Devil of A Time finding anything useful/correct/truthful included dynamic range, sensor noise, and color depth. I was still poking at the things that might go into defining what a "Fat Pixel" camera might be made up of.

Apologies first: For this blog entry I'm going to move very fast and rather deep.

There are nuances and details that can and should be applied to each of the following statements.  Each step can be an entire study unto itself.  As always, don't trust me, but if you must, make sure you verify.

Here is my present state of understanding.  

Sensor Basics ~

~ There is an analog input side of every sensor ending at analog to digital converters (ADC) with the following components -

• Light sensitive photo site
• Analog amplifier
• ADC
  

~ There is a digital output side of every sensor starting at the ADC with the following components -

• Digital image processing chips for stills and video
• Digital image processing software for stills and video
  

~ ISO controls the gain on the _analog_ amplifier that feeds the ADC

~ RAW files are made up of the digitized data spit out of the ADC

~ jpg files are the result of the ADC output _plus_ whatever massaging the manufacturer applies on the digital circuitry in-camera

~ In-camera jpg processing _may_ further increase gain digitally (as a second gain function)

Dynamic Range ~

This is the EV difference between highlight areas with detail and shadow areas with detail.  Sensor noise reduces dynamic range.  Said another way, the quieter the sensor the greater the dynamic range.

In general...

~ The broadest dynamic range is seen at the Base ISO

~ The higher the gain on the analog amplifier (increasing ISO) the lower the dynamic range

Sources of noise ~

~ The analog amplifier (where signal gain is first applied) that feeds the ADC and ADC itself are the primary sources of noise in RAW files (see Canon DSLRs)

~ In-camera jpg processing digital circuit may further increase gain and thereby add noise downstream from the analog circuit.  In general, if there is noise on the output of the ADC, unless there is noise reduction on the digital side, we will see noise out of the digital circuits, too (see Canon DSLRs).

Sensors ~ ISO Variant

These are the traditional CMOS sensors that we've all come to know and love.  Until very recently, these were the only ones commercially available to us.

~ The electron gathering well at each sensor site has these properties -

• Traditional big and somewhat shallow electron gathering well design
• When used at low ISO settings (ie: low analog amplifier voltages) we get -
  
• Broad dynamic range
  
• Slight pixel to pixel variations (subtle noise, if you will)
• Lowest noise/broadest dynamic range tells us what the Base ISO is
• As ISO increases -
  
• Dynamic range decreases
  
• Noise increases... maybe... (see astro-photography reference video below about cases where this is _not_ entirely true)
  

 Recent Sony Design Enhancements ~ ISO invariance

Sensors starting with A6300, A7RIII, A7III, and very obviously the A7SIII, as well as the Sony manufactured MF sensors used in Fuji GFX cameras have _two_ electron gathering wells per photo-sensor site.

~ First electron gathering well at each sensor site has these properties -

• Traditional big and somewhat shallow well design
• Used at low ISO settings
• Broad dynamic range
  
• Slight image to image variations (subtle noise, if you will)
• Lowest noise again tells us what the Base ISO is
  

~ Second electron gathering well at each sensor site has the following properties -

• Narrow but deep well design
• Used for higher ISO settings
• Shows reduced dynamic range
• BUT it shows _less_ noise than the big, shallow electron well
• Highest dynamic range when using this higher ISO than base ISO 1 well-type sets a Second Base ISO
   

Interesting property: ISO invariant sensors tend to show no change in noise as ISO is increased after the Second Base ISO has been switched on.

Note: There are examples of sensors that are ISO invariant from low ISO, such as the Nikon D750 where noise levels do not change from ISO200 on up. See the astro-photography reference video below for the Nikon example.

Comments ~

 Taken as a whole I feel I can begin to understand a few seemingly unrelated things.  Such as -

• Early Kodak CCD MF sensors showed the best image quality at the time.  This, even if they _expanded_/_amplified_ the ADC output from 14bits to 16bits.  At least they started out at 14bits when everyone else was down around 10bits of RAW and/or stuck at 8bits jpg.  I can see where the "Fat Pixel" ideas could stem from in terms of perceived image quality.  What I've learned is that modern sensors can easily outperform the original Kodak MF sensors in every measurable way, _and_ we get ISO flexibility with current sensors where Kodak worked best at Base ISO, period.
  
• I now see (on Photons to Photos - see reference link below) the differences between the old 14bit ADC Canon 5D MkII and 7D sensors and the _12bit_ ADC Sony A6000.  In one release of products we went from "ya, that's not bad but if we're not careful there's loads of noise in the shadows" to "wow! now there's a clean image".  The A6000 has more dynamic range at 12bits ADC and, as hoped for, less noise than the older 14bit ADC Canon sensors.  There is likely something about Canon's analog circuit that was rather prone to the introduction of error.  It's interesting to me that it's only with the release of Canon's new mirrorless cameras that their sensors seem to meet or slightly exceed the early Sony FF sensors.  This tells us something about how much work Canon has done on the analog side photo-site to ADC path.
  
• Looking (again on Photons to Photos - see reference link below) at first generation Sony A7, A7S, and A7R, there's not really all that much difference between these models in terms of dynamic range and noise.  Yes, the A7S has slightly better dynamic range than the A7, but I'm not sure I'll ever actually see or appreciate the extra 1/3stop EV the A7S has.  I've read where we can begin to see differences of 1EV dynamic range, but I haven't verified that for myself.  Coming back to the Photons to Photos information, the A7R looks nearly as good as the A7S.  In the end, if I can't make a great "Fat Pixel" image with any of these three cameras, I'm doing something drastically and dramatically wrong.
• ETTR (Expose To The Right) "works", but not for the reasons we're commonly told.  It "works" because we are able to bring the dark areas up out of the base level noise.  Of course we have to guard against saturating the highlights beyond recovery, but once the data has been collected, it's more flexible in processing than images where the dark areas are down in the noise base of the sensor.
  
• ETTL (Expose The The Left) "works" as expected.  We use ETTL in black and white photography as a "lazy man's way" of guaranteeing as much detail in the highlights as possible.  This can closely emulate film images when processed appropriately.  The highlights are raised in processing and the shadows raise with the highlights. As an aside, I've started to use Zebras to know when the highlights are saturated.  Coming back to ETTL, with quiet sensors we might be able to avoid distracting noise to the shadow areas.  Which leads me directly to the next item.
  
• One of the problems I had trouble understanding why there is so much noise in the shadow areas of severely under-exposed images even at 100ISO where I thought we should see the lowest noise everywhere across a broad dynamic range image.  It turns out the noise is easily explained.  The electron gathering wells aren't able to gather enough information consistently across the wells to make the dark areas appear as smooth as the light areas.  The light simply is not available at those low levels to distribute evenly. Hence noise, even at 100ISO.  To avoid this problem in a single shot, use ETTR or better yet Zebras.  If the dynamic range of the scene is broader than a sensor can handle, there's HDR and image stacking during processing.
  
• Potential recent "Fat Pixel" candidates could be the Sony sensor manufactured Fuji MF GFX cameras.  Their dynamic range exceeds by 1EV the best Sony FF.  The MF Fujis are well above my current Pay Grade and I doubt we'll ever see them go for less than 1500USD used.  Yes, I know.  The 50R is trading hands on the used market for around 2250USD.  It's still too rich for me.  But there is something to be said about well-engineered sensor development and ISO invariant circuitry.  Kodak sensors were never ever close to being this good.
• I now understand why backside illumination of a sensor "works."  By raising the black base to a known level, sensor noise levels are suppressed by starting at zero (pure black) well above the potentially noise inducing analog circuits.  It's a neat trick, actually.  I feel there are some creative solutions being applied to sensor design these days.  This is Fun Stuff.
  
• Pursuing the absolute best color depth, longest dynamic range, and lowest noise images possible requires a static subject, a tripod, setting the camera at its Base ISO 1, and shooting three or four identical images.  Three image stacked low ISO photos "work" because the process averages out the subtle fat/shallow electron well variations.  The final output image quality should easily exceed that of, well, just about anything.  And speaking of which...
  
• Sony has come up  with something interesting in their dual Base ISO sensors.  While destined for video work, I can see where there will be benefits for us stills shooters, too.  Even if we're stuck at 8EV dynamic range at Base ISO 2, I'm still struck by the possibilities of lower than Base ISO 1 noise.  

It's a Crazy Topsy Turvy Counter-Intuitive World out there and I can't wait to see what the Sensor Development Wizards come up with next.

Until further notice I have stopped reading the thoughts, evaluations, and conclusions of the vast majority of popular still-photography related websites.  In general they are at a distinct lack for meaningful data on sensor performance, dynamic range, color depth, and noise characteristics.  I know they're "trying" the best they can, but I've found that I have a difficult (Gear Grinding) time with the Marginal at Best "test" methods and bad data they attempt to analyze and justify their conclusions with.  It's become just "noise" to me.

 

Sometimes a person has to dig deeper to get at truth.

 

References ~ [updated/modified on 4 March (correcting a link to the most important video, the first, and updated slightly on 3 March, 2023]

Here is a video with very clear, concise explanations of Sony sensor behaviors.  

I had to pause the video every other sentence, or so it seemed, so I could stop and think about and think through what was just said.  There is so much _good_ information here.  In fact, _this_ should be the reference for discussion of sensor performance.  I wish more of YouTube was this accurate and informative.  I've found there is too much "squishy thinking" go'n on out there!!!  So this is an absolute breath of fresh air.  

It's easy to see how much this video influenced the organization of my thoughts.  While I knew various pieces and parts of the process, the video helped me organize what I knew into a cohesive whole.  In fact, I've borrowed from the video outline in the writing of this piece because I found it that useful.

Photons to Photos has a great site on measured dynamic range, noise, and sensor performance. Complete with methods and rationale explaining what they do and why. For some reason I've not run into this site until now.  There were many spent hours looking around, reading, and comparing various sensor data.  It's amazing how much a person can learn if they're patient and aware and take a few notes along the way.

Here is a video on ISO invariance and why it's interesting and useful.  There are also comparison images showing how ISO variant and invariant sensors behave differently. 

I often wondered what changed with the introduction of Sony's cameras around 2012.  Reading Jon Rista's explanation from 19 January 2015 - 01:30 PM gives a clue.  While I might not agree that 500nm waffer technology was a problem (except as an example of Canon not keeping up with the latest waffer fabrication trends), the rest of his argument rings true.  Further, Jon Rista gives us a few potential clues about "Fat Pixel" photo-sites from Posted 29 January 2015 - 12:09 AM (pay attention to his comments on photo-site area and electron well capacities). For a Geek who really wants to Geek Out it's very interesting and potentially practical stuff

Lastly, here is a link to a Wiki page where we can see a Sony camera and function matrix.  In addition, Sony has a matrix of ADC bit depth information.  This too is organized by camera and function.  Using this information we can begin to guess how much circuitry is implemented in the various cameras that influence things like read speed and camera capabilities.

To me the most important outcome of all the Geeky Nerdiness is the acquisition of knowledge that can be applied and balanced in the real world when pursuing highest possible image quality.

Update of a common 'net expression from when dinosaurs roamed the earth: Base ISO 1, f/8 and be there.  

If it's dark out, Base ISO 2, wide open and be there.