Friday, April 16, 2021

Super-Resolution - Image Stacking + Sharpening

Previously I covered a simple cubic up-rez + USM "super resolution" technique.  In this blog entry I would like to cover a second "super-resolution" technique.  This involves up-rez'd image stacking.

This all re-started for me when some folks talked about Photoshops "new" function that up-rez's an image.  Some people think it's the cat's meow.  Others point out that Topaz AI somethingorother is better.  And there have been comparisons showing how "good" an up-rez can be these days.  

In contrast, the image stacking approach took the idea that shooting a number of images handheld would cause just enough pixel to pixel displacement that a careful practitioner could average the information when up-rezing each layered stack image and then setting each layer's opacity. The idea tries to emulate in-camera multi-shot sensor displacement and image stacking.  Olympus and Sony implement this feature on some of their cameras.

Here is the approach.

  1. Take a number of handheld images of a scene
  2. Load these images as layers into Photoshop or the Gimp
  3. Cubic up-rez - with an appropriately high interpolation filter sample rate
  4. Align the layers - this can be very tricky, but there is software that can help
  5. Set the Opacity of each layer to average the information
  6. Flatten the image
  7. Unsharp Mask sharpen or use some other image sharpening method

 

NOTE: Remember that I've chosen the Gimp specifically because the software designers have correctly implemented the Cubic interpolation function.  We will select the X/Y resolution of the interpolation filter and it will be properly applied to the image.  

This is very important as I've found that some software packages don't correctly apply the image resolution settings when they perform an up-rez and images can come out "blocky" and "pixilated" as you increase the image dimensions.

Here is what I suggest.  Using the Gimp, select...

  1. Image -> Scale Image
  2. Quality -> Interpolation -> Cubic
  3. X/Y resolution -> 1200 - this right here is the secret to success

 

Here, once again, is the base scene that I will work from.

Base Image ~ "Super Resolution" comparison ~ 2021

 

In the following comparison I show the base image as processed in RawTherapee and with "Capture Sharpen" applied.  

Then I show the Gimp output of a 4 image stack with Image -> Scale image from 6000 pixels on the long side to 9000 pixels (a 2x area increase in size) with light USM (unsharp mask) applied in selecting 1 pixels.  This is followed by the image stack sharpened with a sharpening function implemented in G'Mic called Richardson Lucy, which is much more aggressive than a USM.


NOTE: Some practitioners suggest using as many as 20 or more images to stack, up-rez, and then average.  I have tried this approach and after 3 or 4 images, I can see no improvement in image "resolution." YMMV.

 

Stacked 4 Images ~ "Super Resolution" comparison ~ 2021

 

The image stack approach really seems to struggle to add the expected "resolution" to the up-rez'd image.  The USM image is soft to my eyes, even with just a mild 2x area increase in image size.  This should be "easy", right?  Well it's not.

The Richardson Lucy sharpened image looks pretty good, but it is starting to look "artificial" and "water colory."

Unless I'm seriously missing something, the handheld multi-image stacking approach doesn't quite live up to its initial promise.  It would be interesting to see how this compares with Olympus or Sony sensor "wiggle" in-camera up-rez functions.  Should someone care to share an image or two, I'm all eyes.

What is approach does, however, is provide for very clean, noise-free output.  So, in my way of thinking, there is a definite use for this technique. I have tried this using very high ISO images where there is a ton of noise and the stacked output looked rather nice.  From what I hear, cellphones freely use this approach when making images in dim light.

Thursday, April 15, 2021

Super-Resolution - Cubic and Unsharp Mask image up-rez

I couldn't help but notice that folks are talking about Photoshops "new" function that up-rez's an image.  Some people think it's the cat's meow.  Others point out that Topaz AI somethingorother is better.  And there have been comparisons showing how "good" an up-rez can be these days.

If you know me, you'll likely smile or laugh or possibly cringe when I say I feel there was nothing new under the sun and that new Photoshop and Topaz products are, perhaps, little more than re-packagings of previously existing functions.

 

NOTE: Increasing file sizes does not add information using the aforementioned tools or using the steps described here.  If data isn't in the original file to begin with, it will not be added by increasing the dimensions of the image.  This is potentially important as some software providers imply that Artificial Intelligence (AI) is being used to improve an image in ways that were not previously possible.  This is a demonstrably false implication.  Don't fall for it.

What Adobe and Topaz are doing is simply this.

  1. Cubic up-rez - with an appropriately high interpolation filter sample rate
  2. Unsharp Mask - other image sharpening methods - set to various "sharpening" levels

Knowing these things, we can do the very same using the free Open Source Software the Gimp to demonstrate exactly  what the pay to play companies are selling.

I've chosen the Gimp specifically because the software designers have correctly implemented the Cubic interpolation function.  We will select the X/Y resolution of the interpolation filter and it will be properly applied to the image.  

This is very important as I've found that some software packages don't correctly apply the image resolution settings when they perform an up-rez and images can come out "blocky" and "pixilated" as you increase the image dimensions.

Here is what I suggest.  Using the Gimp, select...

  1. Image -> Scale Image
  2. Quality -> Interpolation -> Cubic
  3. X/Y resolution -> 1200 - this right here is the secret to success

 

NOTE: If you use X/Y resolution of the default 300dpi, the sample rate is too low and your image will be "blocky" and "pixilated" after you increase the image dimensions.  You can try setting X/Y resolution to 600dpi if you like.  It will certainly work for 2x linear file size increases.  I prefer the 1200dpi setting as the interpolation "slices" the filter takes will be 2x finer than 600dpi and 4x finer than 300dpi.  If you don't understand why this would be the case, ask me and I will try and point you to a layman's description of interpolation filters.

Here is the base scene that I will work from.

Base Image ~ "Super Resolution" comparison ~ 2021

 

In the following comparison I show the base image as processed in RawTherapee and with "Capture Sharpen" applied.  Then I show the Gimp output of a simple Image -> Scale image from 6000 pixels on the long side to 9000 pixels (a 2x area increase in size) with light USM (unsharp mask) applied in selecting 1 pixel and 2 pixel mask widths.

 

UpRez'd Single Image ~ "Super Resolution" comparison ~ 2021

 

As you can see, if you start with a "clean file" (ie: well controlled noise) the USM sharpened output looks pretty good.  This is as good, in fact, as anything I've seen from the new Super Resolution products, because, as I said earlier, those products aren't really bringing anything new to the table.

A last note before we move on.  I've found that the Sony APS-C sensors (even the 10+ year old sensors) out perform Canon's current Full Frame sensors at low ISO.  Canon CR2 raw images have a lot more noise in the shadow areas than Sony AWR raw files.

Further, after working with Canon and Sony raw images for well over a decade, I have the strongest impression that a Sony NEX7 low ISO file up-rez'd from 6000 pixels to 9000 pixels on the long side are cleaner and clearer than a native resolution Canon CR2 file of any file dimension, even with a decent "Capture Sharpen" applied.

Heresy, perhaps?  In my case it seems to be the truth.

Tuesday, March 23, 2021

Check out this guy's stack of glass.

 Old lens inventory.

It seems he has a YouTube channel as well. 

I'll never ever get anything close to what that guy has.


Family Portrait ~ Nikon manual focus lenses
 
A few lenses
Many of these have found new homes 
since I took this photo

Sunday, March 21, 2021

Down another rabbit hole...

Shared with little comment...

Aero Ektar lenses on old cameras (yes, I used to own an Aero Ektar and used it on a Speed Graphic, too, but what a BEAST that setup was)

David Burnett on photography

David Burnett photographing the 2nd impeachment hearings

Eric Lindbloom "Angels at the Arno" is a book that was shot using 120 film in a cheap meniscus lens Diana 

David Hamilton's "Venezia"/"Venice" is a book that was shot in a very soft style

A friend sent links and hints to these and other stuff just yesterday.  After wandering around that side of the photo-universe I felt I had to share them in case you're not already aware (I certainly wasn't - not that I know everything - very far from it, in fact).

The passion for photography is clearly evident to me.  Makes me wish Paris hadn't just re-confined for the third time in a year.  I feel stuck along the cote d'Azur and I don't have access to my "things."  I feel I'd like to continue to explore the use of a lens that I modified.  And I'd like to order a few books, too.

I realize this is a First World Problem and that we're all in the midst of something deadly serious (Covid-19).  Hopefully my wife and I will both have our vaccinations soon.

 

Peille ~ 2021

Peille ~ Village Perche'
in the Alps above Menton and Monaco

Thursday, March 11, 2021

Image Sharpening ~ in-camera and Capture Sharpen (RawTherapee)

The prior two winters we in Nice I feel I learned quite a bit about out of focus rendition (aka: "bokeh").  I had a small stack of Nikkor and Takumar manual focus lenses that I'd brought with me.  The project was sparked by my wondering how on planet earth the 85mm f/2.2 SF Pentax had apparently such extreme depth of field, even wide open at f/2.2.  My questions were eventually answered and I felt I'd spent my time well during winter "down time" when there wasn't much to do along the cote d'Azur.

This year I changed things up quite a bit.  I left all my manual focus lenses at home and abstained from dragging the gorgeous low-milage A7 full frame with me.  This year I'm going all AF after having learned that manual glass can be quite heavy and the weight limit on baggage can be breeched if one isn't careful when making an emergency return to Paris via aircraft.

I brought with me two EVF Sony APS-C mirrorless cameras.  One is a NEX7 that a friend sent me and the other is my old A6000.  The A6000 has a 70-350mm f/4.5-6.3 G-Master OSS mounted on it and the NEX7 has mounted on it what turns out to be a very lovely 16-70mm Zeiss ZA f/4 OSS.  The out of focus rendition of the Zeiss lens is to my eyes nothing short of wonderful.  I may have to write something about these two lenses someday, but we'll see.

This has left me with no photography blog project to work on.  The weather is different this year than the last two (colder and wetter).  The pandemic has kept folks locked down (even as they fail to wear masks - is it any wonder that this part of France has blown up "red" on the Covid charts?).  And even though we bought a used car (our first in our nine years of living in Europe), we head out about once a week.  This still leaves a ton of "down time."

My mind ricochets off this and that idea.  Daydreaming, thinking about this and that, and considering my recent realization that Sony has correctly implemented the human perception model for converting color to black and white, something tickled the 'ol brain cells and I got to wondering how "sharp" in-camera jpgs might be compared with AWR (raw) format images processed with "Capture Sharpen" in RawTherapee. 

This was my first pass.

Image Processing ~ sharpen comparisons
 
Here is the full scene

Image Processing ~ sharpen comparisons

Click on the image and inspect it at 100 resolution

It is pretty easy to see that the AWR image processed using "Capture Sharpen" is "sharper" than the out of the camera jpg with "Sharpness" set to 0.

This led naturally to the question of what might things look like if I took the in-camera jpg processing "Sharpness" up to +1?  Here's what I found.

Image Processing ~ sharpen comparisons

Here is the full scene

Image Processing ~ sharpen comparisons

Click on the image and inspect it at 100 resolution

Comparing AWR/"Capture Sharpen" image with the out of the camera jpg with sharpness set to +1 suddenly became much more difficult. At first I didn't notice any difference. It turns out, differences are rather subtle.

My eyes feel strained.  On very close 100 percent full resolution inspection (ie: staring at the screen for minutes on end) AWR/"Capture Sharpen" _feels_ to me "digital" with hard edges and steep light/dark transitions.  The in-camera jpg processing _feels_ much more like film to me.  There is a nice "roundness" to the sharpened image. 

I will continue to shoot AWR because I like the flexibility in image processing.   If I really want a film like "roundness" to my AWR, I can apply a light unsharp mask or turn down the sharpening parameters of "Capture Sharpen".  

There may be times when in-camera jpg processed images are more than adequate.  If I set the imaging storage to AWR + jpg I would have the best of both worlds, right?  

Which might lead to an interesting new project.  What?  I have no idea.  I seem to have plenty of time on my hands to think about it, though.

Thursday, February 18, 2021

Small sensors and reducing file sizes...

A friend sent me a Sony HX90V 30x point and shoot camera to play around with.  He said there's nothing wrong with carrying a 30x zoom "toy camera" since it fits nicely in a pocket.

The HX90V small sensor has a narrow dynamic range and tends to be rather "noisy", even at low ISO.  These things, compared with the APS-C Sony mirrorless cameras.

Just for grins, I decided to see what would happen if I reduced the file sizes and to see if or where image quality might come close to the APS-C performance.

Here is what I found.

[As always, click on the following image and enlarge it to 100 percent to observe any differences]

 

Nice 2021

 

The images down the center of the panel are 100 percent full resolution crops of their respective base images.  The superiority of  the APS-C system in terms of pure image quality is evident.

Looking at the downsized images at 100 percent (the images down the right side of the panel) show things have pretty much "evened out."  The HX90V photos look pretty good.  The only somewhat minor differences being in color rendition.

Which goes to show that a little 30x zoom point and shoot camera is just fine for sharing images across the internet.

And if you don't live or die by side by side comparisons with other systems, with proper care and handling the little HX90V Sony can turn out rather acceptable images, even for up to A3 size prints.

Nice 2021

Sony HX90V image from
a walk around Mont Boron, Nice, France


Wednesday, February 03, 2021

It is nice in Nice...

Yesterday was a glorious day here in Nice.

My wife and I went for a nice walk along the sea.  I had the Sony HX90V with me and shot a few images.


One was a set of images to be used to create a panorama.
 
Just this morning I stitched the image using Hugin.

I think it came out well enough.  It is over 10,000 pixels on the long side and contains a nice amount of detail.  It should print very large with ease.

All this from a little Point and Shoot camera.  You know the type.  It's the kind of camera that very few people carry these days.
 
Yes, I am now officially well behind the curve when it comes to using "new" cameras and lenses and keeping up with the "in" crowd.
 
 
Nice Port ~ 2021