Random Thoughts on Photography [8] ~ color management and color "science"

 

I've been thinking and poking and looking at the topic of color management in light of manufacturer claims.  Certain camera companies claim to have a "secret sauce" that is called "color science."  I wanted to consider this and see if there might be ways of pollinating these much vaunted "color sciences" across other manufacturer flowers, er, cameras. 

Background ~

I smile as I remember the battles that were waged on-line over who had the "better color science", Canon vs Sony.  I still read comments from Fuji users who've fallen in love with Fuji's film profiles.  Leica users wax poetic about a certain "Leica look."  And for years I've read how Hasselblad has the "color science" of any of the commercial camera manufacturers.

Is it real or is it marketing blather?  Perhaps only real "color scientists" know for sure.

On a practical level and before I dove into this topic, what I experienced was that my old Canon images are always redder than my very neutral looking Sony output.  Reviewing carefully Fuji X or GFX images I can't see where they are demonstrably different nor better than any other products on the market.  I can't look at an image and "see" any Leica Magic coming out of Leica products.  Nor am I convinced that Hasselblad has a better understanding than anyone else in the industry about how to make colors.

Yes.  I'm a "color science" skeptic. Why?  Well, it's a very long explanation.  So sit down, strap in, and hold on tight.  Here I go.  Yet again.  Bashing some drum or other.  This time it's "color science."

In-Camera Pipeline ~

Here is how I understand digital imaging to work, from start to finish.

1. Light hits a light sensitive diode
   
2. Diode spits out a tiny electrical signal
3. Hardware applies gain to amplify the signal (ISO sets the amplification level) 
4. Amplified signal enters an analog to digital converter (ADC)
   
5. Digitized information exits the ADC (may receive additional signal processing as in the cases of extended ISO settings)
6. Now there is a fork in the process ~
   
1. Digital information passes through an ASIC that processes digital data and write it into an in-camera generated JPG file
2. Digital information is written into a RAW file

We now have two file possibilities, RAW and/or JPG. 

A JPG file is the result of what the manufacture implements, purely and simply.  It has passed through an in-camera de-mosaicing algorithm of the manufacturers specification as well as other imaging algorithms (where various "looks" can be applied).  

JPG image processing decisions have been made by the manufacturer, so if there's a special "color science", it can be applied here.

Off-Camera JPG Pipeline ~

It is potentially important to note that further processing on a JPG will start with the  limited color range of 8 bits as found in the file. Heavy processing can distort colors, destroying any special "color science" that had been "baked" into the JPG file.

Off-Camera RAW Pipeline ~

RAW files require additional processing.  If your software has the possibility of displaying true RAW file information, you'll see a strongly green cast mushy image sort of thing that has red and blue dots evenly sprinkled around the image field.  In short, it's an ugly mess and it's easy to see why we need software to sort things out for us.

Here is the Bayer RAW format pipeline.

1. RAW file is loaded into software
2. De-Mosaic is most-often the very first function performed on a RAW file, where individual "pixels" of RGGB (red, 2x green, blue) information is processed using surrounding "pixel" data to calculate color and luminosity such that there is now a different kind of information (de-mosaic calculated) for each "pixel", changing each and every original red, 2x green and blue locations into full color with brightness
   
3. De-Moscaic'd images still look ghastly as the colors are still "off", so...
4. Software "massages" a de-mosaic'd image and displays it to the user
5. User can now begin performing additional modifications using the tools the software provides

Where's the "color science" in RAW image processing?

There are a couple candidates for where a manufacturer can add their own "special sauce."

The first place we can ask "what has a manufacturer" modified? is the de-mosaic stage.  From what I've read, this step would be an incredibly difficult place to add a dash of "special sauce".  The algorithms are complex and varied.  Have a look at the de-mosaicing possibilities in the Open Source Software RawTherapee and you'll see what I mean.

It turns out there are very good reasons to use the various de-mosaicing approaches, but if you are a RentWare user, I suspect you have zero control over the de-mosaic stage.  Please tell me if I'm wrong.

In any event, I feel that "color science" is not applied at the de-mosaicing stage.

The next stage is where things can get a little more interesting.  This is where camera (better said, sensor model) specific color management comes into play.

Some RentWare and Open Source software use .dcp format files to store camera-specific color corrections.  Hasselblad embeds camera-specific color corrections in proprietary XML formatted files.  I'm not sure what Sony, Canon, Nikon, Fuji, Olympus, or Panasonic proprietary software use, but rest assured, both de-mosaic and color management functions are "baked" into them.

In nearly all cases, camera-specific color corrections are hidden from the user.  However, there are instructions on how to generate .dcp color management files for RawTherapee.  Reading these instructions is insightful and related directly to the question at hand.

Taking industry standard color charts, camera model to camera model variations can be corrected.  If you use RawTherapee, for example, have a look at the Color Management tool and compare the colors between "Camera Standard" and the automated camera specific colors and no corrections.  The differences are illustrative.  Then look at properly color managed files from different cameras and tell me where you see any differences.  Using non-manufacturer software tends to eliminate the effects of any "special sauce." 

If a camera manufacturer is to inject some special "color science" during RAW image processing the color management stage would be an excellent place to do so.

What Else? ~

It is _after_ the de-mosaicing and the subsequent color management stages that color look-up tables can be applied.

Color grading (not to be confused with color management) can be a one button exercise in many software.  Many times these can be implemented as LUTs.  Video software as well as still photography RentWare come with many LUTs pre-installed.

With proprietary and RentWare software you get the de-mosaiced, color managed file displayed after RAW file import.  Then there is this  thing that they provide that adds a special "look."  For Sony it's things like "Neutral", "Standard", "Vivid", "BW", etc.  That's the color look-up table (LUT) stage.

What's potentially interesting here is that as a user you can generate your own LUTs.  Of course you can buy them, too.  What's nice is that the user controls the output.

Last Question ~

Might it be possible to somehow "borrow" from one software to another and preserve a manufacturers recipe?

Maybe.  It depends on several things.  If we can import an industry standard color diagram and have a manufacturers color management applied to it, then yes, there's a possibility we can duplicate a "special sauce" and move it to another software.

Yet I have my suspicions.  I see manufacturers have generated closed system software where the early image processing steps are hidden from the user.  We don't know exactly what parameters are being used and we're in the dark about many potentially important details.  So, in the end, I'm not sure how successful we can be, but we might be able to get surprisingly close.

Another approach could be to open a manufacturers proprietary software with an image who's processing we like, then next to it open another software and change processing settings until the image look the same, then save the actions or generate a LUT.  This is much more "hit and miss" and would not be very accurate.

By hiding what manufacturers are doing behind the wall of proprietary software they can make any marketing claim they want.  To make matters worse, RentWare does exactly the same thing. There is no way of verifying any of these claims.  Further, such claims can then be picked up by communities of photographers where they can take on a life of their own.

On the other hand, what I'm experiencing these days is that by more deeply understanding each and every step in a RAW file processing pipeline that I am at liberty supported by increasing knowledge to create images that are pleasing to me.  Matching one companies "color science" to something I'm working on has become a fun and interesting challenge and I'm very happy when I get very very close, if not precisely spot-on, to something that's been "secret sauced" hidden behind a proprietary/marketing "color science" wall.