Tech info: Dynamic Range of DSLRs

[Daniel Browning]

there is a difference between noise reduction when a number of images are averaged together and the noise reduction resulting when image is resized.

Thanks for the explanation, Joofa. Would you consider posting some specific example images where the SNR improvement was less than the square root of the area ratios?

 

[Daniel Browning]

My reading of paragraph (a) at the top of this page
http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p4.html
is that Emil draws a clear distinction between temporal and spatial averaging. I wasn't able to find anywhere he confused them.

I think what Joofa is saying is that Emil believes both are capable of reducing noise by the square root of the size difference in sampling.

 

[Daniel Browning]

What I was referring to was that from certain ISO, like 1600 the read noise starts affecting the effectivnes of applying analogue gain and at some point there is no benefit of doing so. Compression artefacts aside, at that point there is also no benefit of applying digital gain as this can be done just as effectively in post processing. Hence introduction of "extended" gain settings in Canon DSLRs.

The "extended" gain settings in Canon DSLRs are not metadata.

If they were, I would be able to use auto exposure, flash exposure compensation, lcd review, etc. without sacrificing dynamic range in my Canon cameras.

 

[Pawel Achtel]

The "extended" gain settings in Canon DSLRs are not metadata.

If they were, I would be able to use auto exposure, flash exposure compensation, lcd review, etc. without sacrificing dynamic range in my Canon cameras.

Correct.

 

[Dj Joofa]

My reading of paragraph (a) at the top of this page
http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p4.html
is that Emil draws a clear distinction between temporal and spatial averaging. I wasn't able to find anywhere he confused them.

I could have read his pages wrongly. However, reading his pages I could sense it as an underlying assumption -- if you consider the degenrate example of adding noise to a constant image for analysis he presented; his statements such as on the ones below from his webiste:

"Bottom line: At the cost of having half the linear resolution, the superpixel made by binning together a 2x2 block of pixels has twice the signal-to-noise ratio,"

"If one downsamples an image properly, one decreases the resolution, and noise decreases in proportion to the linear change in image size.";

and, additionally, lack of analysis regarding the differential rate of the signal change for SNR formulation, etc. For e.g., best SNR in simple average-based downsampling is related to second derivative of signal intensity.

Thanks for the explanation, Joofa. Would you consider posting some specific example images where the SNR improvement was less than the square root of the area ratios?

Daniel, you are welcome. Yes, I shall reply to your post tomorrow.

 

[Dj Joofa]

Thanks for the explanation, Joofa. Would you consider posting some specific example images where the SNR improvement was less than the square root of the area ratios?

Hi Daniel,

It is easy to figure out when SNR would be less in resizing based upon averaging. Suppose you take an image and average each 2x2 pixels into 1 pixel, then you get an image 1/4 in size of the original image. In this case, spatial pixel averaging results in good noise reduction and many people erroneously conclude that the process could be extended to averaging even larger number of pixels, i.e., 4x4->1, 8x8->1, 16x16->1, etc. But there is a problem, though noise is being reduced, the image is also becoming more and more blurry.

The problem is that the assumption is that the mean square error used in formulation of SNR is only dependent upon only on noise reduction, where as, it has an additional dependency on the signal degradation due to smoothing, which is frequently ignored (the bias term in my previous message). In the worst case just consider, that at each pixel position you average all pixels in an image and you end up with a flat, constant image, where each pixel has the same value. That is pathetic SNR.

On the other hand, it does not matter how many images you average in time to get a cleaner image, if it is considered that all those images are of the same static shot where each pixel had a fixed true value, but was corrupted by noise (hopefully zero mean). Averaging a constant number is the same as the constant number, so the original signal is *not* blurred (bias is zero), however, the noise is reduced significantly, and the SNR improves.

 

[Daniel Browning]

Suppose you take an image and average each 2x2 pixels into 1 pixel, then you get an image 1/4 in size of the original image. In this case, spatial pixel averaging results in good noise reduction and many people erroneously conclude that the process could be extended to averaging even larger number of pixels, i.e., 4x4->1, 8x8->1, 16x16->1, etc. But there is a problem, though noise is being reduced, the image is also becoming more and more blurry.

What do you mean, "blurry"? Do you mean if the resampled version and the original version are compared at the same output size, the resampled version will look more blurry?

 

[jbeale]

averaging pixels throws away information

averaging pixels throws away information

...But there is a problem, though noise is being reduced, the image is also becoming more and more blurry.

The problem is that the assumption is that the mean square error used in formulation of SNR is only dependent upon only on noise reduction, where as, it has an additional dependency on the signal degradation due to smoothing, which is frequently ignored (the bias term in my previous message). In the worst case just consider, that at each pixel position you average all pixels in an image and you end up with a flat, constant image, where each pixel has the same value. That is pathetic SNR.

You are quite right, of course, that if you compare the SNR of two images at different resolutions, you have to make some assumptions about what exactly is defined as the signal and what is the noise. I think often people separate out the spatial frequency response of a system into the MTF, and then calculate overall SNR without regard to frequency (eg. over all frequencies in a given image, or in fixed frequency bins).

 

[Dj Joofa]

What do you mean, "blurry"? Do you mean if the resampled version and the original version are compared at the same output size, the resampled version will look more blurry?

Assuming an additive noise model on the input, the output of a linear time invariant filter is the sum of the the action of the filter transform on the true input signal and the noise, separately. Even, for a fixed downsampling size, say 4:1, in simple average based filtering, the output resulting from the input noise term gets smoothed and becomes less as the size of the filter increases. However, the size of the averaging filter also has a smoothing effect on that part of the output coming from pure input signal. Therefore, a larger filter term results in more smoothed or blurry signal.

 

[Daniel Browning]

the size of the averaging filter also has a smoothing effect on that part of the output coming from pure input signal.

Thanks again, Joofa. Now I'm pretty sure that I did understand you all along. You're saying that downsampling an image to a lower resolution makes it have lower resolution.

On the surface of it, that's pretty obvious. But underneath the surface what you're saying is that the lower resolution means lower signal. I've never heard of that definition of SNR before (I use ADU per pixel), but it makes perfect sense.

Here's an example to bring it all together (I hope).

Camera A has 2K pixels and 2048:1 SNR in ADU/pixel (11 stops/pixel)
Camera B has 4K pixels and 2048:1 SNR in ADU/pixel (11 stops/pixel)
Camera C has 4K pixels and 1024:1 SNR in ADU/pixel (10 stops/pixel)

Four images with Camera A, stacked, yields 12 stops/pixel SNR at 2K resolution.
One image with Camera A, unchanged, yields 11 stops/pixel SNR at 2K resolution.
One image with Camera B, resampled, yields 12 stops/pixel SNR at 2K resolution.
One image with Camera C, resampled, yields 11 stops/pixel SNR at 2K resolution.

Of course, that's assuming random, uncorrelated noise and perfect brick-wall anti-alias filters. If you throw real-life anti-alias filters into the mix, Camera B will have higher detail at 2K. Combining that with your "SNR per resolution" model, we get:

Camera A: 11 stops per 2K resolution (2.3 MP).
Camera B: 12.3 stops per 2K resolution (depending on anti-alias filter).

If you resample beyond the imperfection of the anti-alias filter:

Camera A: 12 stops per 1K resolution (0.59 MP).
Camera B: 13 stops per 1K resolution (0.59 MP).
Camera C: 12 stops per 1K resolution (0.59 MP).

Camera A: 13 stops per 0.5K resolution (0.15 MP).
Camera B: 14 stops per 0.5K resolution (0.15 MP).
Camera C: 13 stops per 0.5K resolution (0.15 MP).

Camera A: 14 stops per Youtube resolution (0.04 MP).
Camera B: 15 stops per Youtube resolution (0.04 MP).
Camera C: 14 stops per Youtube resolution (0.04 MP).

One of these days I'm planning to make a post demonstrating ISO 1.6 million using my 20D. It really shows how resampling trades SNR for resolution.

 

[Dj Joofa]

Hi Daniel,

... perfect brick-wall anti-alias filters.

(1) It is easier to reproduce the effect of a sharper brick-wall type filter by first using a less sharp anti-alias filter, then oversample, and then downsample. (So Red would have a natural advantage here if targeting HD type resolution.)

(2) As soon as you downsample (or for that matter upsample) you loose the time/shift invariant feature of a linear system.

 

[sander kamp]

Why? RedCode does not seem to have visible artefacts like MPEG does and it is basically visually lossles to my eye :blink:

It also does not seem to affect resolution or dynamic range, so a comparision of uncompressed and compressed would be purely academic.

How do you know? As far as I know there has never been uncompressed RED footage published so it is hard to tell if there would be any difference. I can tell though that dark, grainy images looks considerably more muddy on the RED than on the my DLSR (D200), which makes me think the compression can't keep up with the noise in the image.

 

[Pawel Achtel]

How do you know? As far as I know there has never been uncompressed RED footage published so it is hard to tell if there would be any difference. I can tell though that dark, grainy images looks considerably more muddy on the RED than on the my DLSR (D200), which makes me think the compression can't keep up with the noise in the image.

The artifacts are so minor that, for all intents and purposes, the image is visually lossless. Visually the compression does not seem to affect the sharpness, detail, smooth gradations or dynamic range.

Artifacts at the noise level are somewhat irrelevant simply because they are at the noise level. Those artifacts are not in any way more problematic than the noise itself, specially that they do not show in a objectionable form (similar to grain).

Also, remember that most DSLRs apply noise reduction in the camera.

And lastly, Red uses linear 12 bit quantization. DSLRs use 14 bit quantization. You are more likely to see the quantization noise than compression.

I am convinced that improvements in noise levels and increased quantization would be much more beneficial than less compression.

In fact, the way forward would be to offer even higher levels of compression ratio at increased spatial and temporal sampling frequencies and at a higher quantization that will also contribute to increased DR.

Much of the Red's success and strength is due to brilliant compression applied where it is most beneficial to the end result. Remove that and you end up with another DALSA and its problems :bleh:

 

[sander kamp]

The artifacts are so minor that, for all intents and purposes, the image is visually lossless. Visually the compression does not seem to affect the sharpness, detail, smooth gradations or dynamic range.

Artifacts at the noise level are somewhat irrelevant simply because they are at the noise level. Those artifacts are not in any way more problematic than the noise itself, specially that they do not show in a objectionable form (similar to grain).

Statements like that just beg to be disproved. As I said, unless you have in some way access to uncompressed RED footage there is no way to know.

I did however do a small test with my RED compared to my D200. Same lens, same shutter speed, same iso, both shot RAW. The nikon image is a bit unsharp because I couldn't eye focus the manual lens - proof how important cine lenses and focus tools are. The images also have a different feel to them and are hard to match in color. But my opinion it is clear to see that the Nikon image shows more detail in the dark areas, showing objects that the RED sometimes totally misses such as the house above the white house on the right.

Are these differences due to compression? I don't know. But to my eye the RED has a more blotchy, compressed feeling while the Nikon is more clean, showing more regular noise. This becomes more apparent in motion, where jumping around artifacts distract more than regular, grainy noise.

On a separate note: notice how much nicer the RED the streetlights handles. Where the nikon shows multiple halos around the lights the RED image is nice and clean.

 

[Dj Joofa]

As far as I know there has never been uncompressed RED footage published so it is hard to tell if there would be any difference.

Yes, this is an important point. Additionally, Red is mildly compressed (about 10:1 I think), therefore, the image is not degraded a whole lot in the first place after compression. A real test for Red compression would be if they were to do compression on the order of 300:1 - 500:1, which is done for certain applications out there. Also, as has been posted several times on this forum, the compression is (a variant) of JPEG-2000 flavor. I don't think Red is going to come up with a whole new algorithm for compression and it will be more playing around with the parameters of existing compression algorithms.

 

[GlennChan]

Uh... I think they did come up with a whole new algorithm for compression. Redcode is designed to work on RAW images, not RGB. Compressing RAW is more efficient because you don't have the signal processing introducing additional information to compress (or throwing information away).

2- CML has some example stills comparing Red uncompressed versus compressed. Keep in mind that it is an EARLIER version of Redcode RAW and a prototype build of the camera... so current results will be a bit different.

 

[Dj Joofa]

Uh... I think they did come up with a whole new algorithm for compression. Redcode is designed to work on RAW images, not RGB. Compressing RAW is more efficient because you don't have the signal processing introducing additional information to compress (or throwing information away).

Obviously, Red would be in a position to comment better here. However, I am not sure that is the case of a whole "new" algorithm, because blindly looking at the problem it is just data compression and you want to remove redundancy, whether the data is Raw or RGB. Of course, one form of data or the other may have more redundancy than other, and it may compress better than other, but that was because of inherent nature of more redundancy in the data, and not necessarily a better compression algorithm.

Here is my take -- At the broad level there are 3 main ways of doing "lossy" compression:

(1) DPCM
(2) Transform coding
(3) Subband coding

And, all 3 of them related deep down there -- and I have to repeat, they are related in a sense. I think Red has opted for Subband Coding. At the top-level you have to minimize the datarate resulting from individual bitrates from each subband. What you mentioned as adapting for Raw vs. RGB would be adjusting how to change parameters in a way that different subbands may offer less bitrate for the same quality, or better quality at the same bitrate. But, the problem is still a subset of the domain of subband coding.

 

[Pawel Achtel]

Statements like that just beg to be disproved.
...
Are these differences due to compression? I don't know. But to my eye the RED has a more blotchy, compressed feeling while the Nikon is more clean, showing more regular noise. This becomes more apparent in motion, where jumping around artifacts distract more than regular, grainy noise.

You haven't disproved anything, in fact your sample images support exactly what I said.

The very fact that you see the difference only around noise floor stretched 2 stops is a clear indication these are not compression artifacts whatever you see there.

You are looking at read noise from the sensor and quantization noise, not compression artifacts, which would equally show in mid tone and highlight details.

 

[GlennChan]

However, I am not sure that is the case of a whole "new" algorithm, because blindly looking at the problem it is just data compression and you want to remove redundancy, whether the data is Raw or RGB.

I think there are a lot of compression tricks that can be applied, some of which are not obvious and some of which are secret sauce.

For example, if you have a 4:4:4 RGB image and convert it to Y'CbCr with the co-efficients for Y' being 0.5 0.25 0.25 (reversible color transform in JPEG2000)... this *adds* information but ultimately improves compression efficiency. This is because real life does not have very many fully saturated colors... the distribution of colors stacks heavily towards neutral/achromatic colors. There's lots of weird stuff like this which can improve compression efficiency.
MPEG4 intra for example has some prediction scheme built into it. So there's a wide variety of compression tricks out there... and presumably why Red took its time in nailing down Redcode.
They can also tailor their compression to what their FPGA is capable of doing. This is not consumer-quality compression, where they stick to what's possible on small ASICs/FPGAs. DV for example compromises compression efficiency heavily to reduce cost... 4:2:2 would be more efficient compression but more expensive (because the DSP has to handle 4+2+2/4+1+1 times the pixels). The DV spec, I believe, also offsets the Cr and Cb relative to each other so they can avoid sticking a delay into the DSP.
DV and Redcode have comparable reductions in bandwidth... yet Red is clearly far superior in compression quality.

2- Redcode RAW is also 12-bit, not 10-bit. Perhaps only Graeme can tell the difference (I don't think I'd be able to tell), but I suppose it would be a technical difference between Redcode and other compression schemes.

And I believe Redcode is designed so that it keys easily... presumably, this is why they released the comparison stills which you can see on CML to figure out if compositors were bothered by Redcode compression (Red's early version of it anyways).

---
The bottom line:
The best thing to do is to test the end result.

Right now there is no uncompressed port so you have to judge the system as a whole (camera + compression). As a whole, I like what I see and think that the Red keys way better than some other uncompressed cameras on the market (e.g. Viper).

 

[Dj Joofa]

I think there are a lot of compression tricks that can be applied, some of which are not obvious and some of which are secret sauce.

Yes tricks and sauces, but not a new algorithm per see at the top level. There is little room for a new algorithm at the top level as the field is already pretty saturated. Therefore, newer approaches have to focus on two things:

(1) determination of the best parameters to work with, and,
(2) Implementation of theory into a practical, workable hardware/software device.

Implementation of even known theory is not easy. And this is where Red has done well.

For example, if you have a 4:4:4 RGB image and convert it to Y'CbCr with the co-efficients for Y' being 0.5 0.25 0.25 (reversible color transform in JPEG2000)... this *adds* information but ultimately improves compression efficiency.

RGB->YCbCr transform is done to decorrelate data. It is not the best decorrelating scheme out there, but works reasonably good, and has historical preference, such as separation of signal into luma/chroma flavors.

Decorrelation is an important essence of data compression.

MPEG4 intra for example has some prediction scheme built into it.

Prediction is done to reduce variance via data decorrelation. These are all the same and known ways of doing data compression.

and presumably why Red took its time in nailing down Redcode.

Implementation issues.

They can also tailor their compression to what their FPGA is capable of doing. This is not consumer-quality compression, where they stick to what's possible on small ASICs/FPGAs ....

These are all implementation issues.

2- Redcode RAW is also 12-bit, not 10-bit. Perhaps only Graeme can tell the difference (I don't think I'd be able to tell), but I suppose it would be a technical difference between Redcode and other compression schemes.

Some compression standards (e.g., JPEG likes 8 and can't seem to remember 11 or 12 bits?) specify the bit depth they want to work with. However, that is an implementation issue.

And I believe Redcode is designed so that it keys easily... presumably, this is why they released the comparison stills which you can see on CML to figure out if compositors were bothered by Redcode compression (Red's early version of it anyways).

Red has hyped already existing technologies such as compression, deBayering, wavelets, etc. Suddenly, you have a lot of people talking about such terms. As MMost has pointed out in a different thread there is a danger of people not properly understanding many of the inherent notions, and more over, the cumulative effect of public attribution of these terms and their progress with a particular company.