HDR Mode...

[Sanjin Jukic]

DxOMark Sensor metrics and photospace

DxOMark Sensor metrics and photospace

To give you some more food for thought in thinking about HDR:

All you need to know about the DxOMark Sensor scale

DxOMark Sensor metrics and photospace

DxOMark Sensor is an objective scale designed to simplify comparison of digital camera RAW-sensor performance in typical,
real-life photographic scenarios.
To build this scale, we have identified key areas of real-world photography that are as different from one another as possible,
and yet taken together, generally describe the entire photo space. The three photographic scenarios and their corresponding metrics
that we have selected are: landscape photography (maximum Dynamic Range); studio & portrait photography (Color Depth);
and photojournalism & action photography (Low-Light ISO). The DxOMark Sensor score itself reports an overall Image Quality
performance value that is computed from these three metrics, Color Depth, Dynamic Range, and Low-Light ISO.

All three metrics are extracted from the Image Quality database camera RAW-sensor data.
On the DxOMark Sensor scale, the higher the score, the better.
(The scale is kept open, anticipating that sensor technology will continuously improve over time.)
The three scenarios and their associated metrics are described in detail in the following pages.
Note:
For all bar graph representations of the DxOMark Sensor scale and Metrics across the site, we utilize the minimum
and the maximum value found for each metric from among all cameras tested.
The value on the right indicates the actual maximum value for the considered metric,
and the value reported on top of the bar is the actual value for the selected camera.
The maximum (right) and minimum (left) bar ends are set to the actual measured values plus or minus 10% of the difference between
actual maximum and minimum measured values for all cameras.

LINK>>>

 

[Rob Ruffo]

That would be one way to do it.

Jim

Photosites provide voltages - another way would be to simply divide voltage numbers by fractions. The analog info at the photosite unprocessed level is like 'super-raw', so there would be more latitude there to do this than at the recorded image level.

Yet another way would be to divide up by time. So... We let, say, 10 milliseconds go by, record the voltages from light accumulation on the photosites, convert to a video frame for our HDR frame set. We then let another 10 milliseconds go by, record the cumulated voltages from the new total (= 20 milliseconds exposure to light so far), 10 more, and we have the result of a 30 millisecond exposure and so on, potentially all the way up to the exposure for a global shutter (24th of second for 24p) in any degree of increments, limited only by the recording speed of the media. Using this method your exposure lattitude would be infinity.

The only problem would be that less-exposed areas would show less motion blur than longer exposure areas when the HDR was later combined. But... The Epic has a motion-sensing device, so some of this misiing motion blur, at least what is attributable to camera motion, could be redrawn using some kind of in-camera or post method, not to mention that vectors could be determined based on some kind of Twixtor-like algorythm that knew which pixels were from shorter exposures in the HDR stack, and would compensate accordingly.

 

[Rick Burnett]

Anyone have some good links on the physics of photosites? I find a lot of pages, but I'm looking for actual circuitry (even if hypothetical) to study. Unfortunately, in engineering school, this is a topic that was never even touched on lightly.

 

[paul gill]

really cool idea!

so in the case of the framerate subsampling idea, the 2/3 is gonna have a much higher dynamic range than the s35, which is a bit annoying, but i suppose the 2/3 would be better used for hdr post stuff due to the bigger depth of field (for example as a source of hdr reflection maps), and lower cost to make it a prefect auxillery camera. But for me who was planning on just the s35 and not also the 2/3, do i have to think again!

sorry if this has already been discussed just havent the time at the moment to read the whole thread.

 

[Rob Ruffo]

really cool idea!

so in the case of the framerate subsampling idea, the 2/3 is gonna have a much higher dynamic range than the s35, which is a bit annoying, but i suppose the 2/3 would be better used for hdr post stuff due to the bigger depth of field (for example as a source of hdr reflection maps), and lower cost to make it a prefect auxillery camera. But for me who was planning on just the s35 and not also the 2/3, do i have to think again!

sorry if this has already been discussed just havent the time at the moment to read the whole thread.

Thanks! - some of this is patent pending by us, but I seriously doubt Red is infringing on our patents and copyrights. If they are they are doing so completely unknowingly/by coincidence which is part of why I post this here.

 

[Gavin Greenwalt]

Or it requires a specific step in the processing chain (like debayer).

I for one would be surprised if it were baked-in in the the sense that it irreversably altered the raw sensor data. I suspect it's more along the lines of taking the additional frames, specific metadata, etc...

In that sense it may not be the exact same as non-HDR RAW data (ie. if mutiple frames are read without an itervening reset), I suppose you could say it might be a baked-in change however.

-sc

You're right. I guess if they use the bursts-of-bursts method all of the bursts could be stored (assuming you have an epic capable of recording full framerate.) But on scarlet you wouldn't have the electronics capable of recording the full burst-capable framerate.

Not baking it though would in bursts-of-bursts hdr be pretty limiting as well. After all you would probably have underexposed shots with too fast of a shutter.

 

[C_Skaff]

Canon definitely does not use metadata iso in its cameras. Red definitely does it its Red One.

(Sorry for being of topic)

Does that mean that when you change the ISO on a Canon... you actually change the physical sensitivety on the sensor? I thought a sensor had a linear response to light and that it is what it is?

Or does it mean that when changing ISO its actually just a Digital Gain/Exposure that might aswell have been done in post?

Or is it in fact a "analog gain" inside the camera before the sensor-data is translated into data?

 

[Steven Caesare]

(Sorry for being of topic)

Does that mean that when you change the ISO on a Canon... you actually change the physical sensitivety on the sensor? I thought a sensor had a linear response to light and that it is what it is?

Or does it mean that when changing ISO its actually just a Digital Gain/Exposure that might aswell have been done in post?

Or is it in fact a "analog gain" inside the camera before the sensor-data is translated into data?

The Canon applies analog, digital, or a combination of both depending on the setting.

Search for Daniel Browning's posts on this as he's documented it...

-sc

 

[Rick Burnett]

So I did take a look at the posts by Daniel Browning, and some of the papers he references. One thing I am trying to understand is how noise comes into play both in both analog and digital gain. I still don't understand how applying an analog gain to sensor output data doesn't just amplify the noise in the CMOS circuit? I can see that it could help prevent digital gain which would amplify the noise floor in the actual A/D converter, but if the sensor is only so sensitive, how does amplifying it make it more sensitive? You can't create data from nothing. If the sensors were in fact that much more sensitive as I said you could just use a larger A/D converter with more bits to capture that wider range. I think in ARRI's design there is a lot of other magic happening that is NOT in their diagram to get their extended detail IMO.

 

[jbeale]

I still don't understand how applying an analog gain to sensor output data doesn't just amplify the noise in the CMOS circuit? I can see that it could help prevent digital gain which would amplify the noise floor in the actual A/D converter, but if the sensor is only so sensitive, how does amplifying it make it more sensitive?

You are correct, any amplifier will always add noise. But it is still a net win if the increase in gain overcomes the input noise (including quantization noise) of your A/D. I believe that is the case with most DSLRs, at least up to some "reasonable" ISO. The latest DSLRs include extreme ISO settings which I think are more about marketing than about image quality.

 

[Daniel García]

Just to clear up some ideas.

Just to clear up some ideas.

Man, talk about wild speculation and couch theorists. Funny, isn't it?
Ok, if you want to clear up confusion and get your definitions straight, I suggest getting "The HDRI Handbook". It's been a loooong time since the Flickr HDR boom and I'm surprised that so many people on these forums still have the "stupid-x-steps-howto-HDR-tutorial" predominant idea of identifying HDR acquisition with the excessive tone-mapped look. Even experienced and respected people are having big conceptual crisis here.
HDR just refers to the adquisition stage, where by some method, be it exposure bracketing, multiple gain techniques, or special cameras, more dynamic range than ordinary can be captured. Ideally, this captured data is scene-referred rather than display-referred, it represents the original light levels present in the scene linearly, without gamma curves or other manipulation.

This data has multiple applications and what you do with it depends on your needs. You can use it to measure real light levels if you are an architectural light planner or something like that. You can capture spherical light fields (light probes) and use them to illuminate a virtual object from all angles with the original scene's real light. You can even relight an actor's performance if it was captured on Paul Debevec's Light Stage.
If you want to produce a direct photographic output, you have a problem. The captured data has to be fitted (mapped) somehow to the inferior dynamic range of the display/output medium. To do this, you use an algorithm, called a Tone Mapping Operator (TMO). There are multiple of these algorithms, and most HDR-processing software offer varying implementations on the few published ones (usually as SIGGRAPH white papers). A simple curve can be considered one of the simplest TMO's, which assigns values to the original data according to their absolute levels only, but most TMO's developed for HDR images take into account some model of human vision where the pixel value depends on its contrast with the spatially neighboring pixels. They usually try to amplify the microcontrast of small-scale details without compromising the overall tone relationships within the scene, but the misuse of these features has lead to a ton of surreal-looking halo-ridden images with parts of the sky darker than the ground shadow details. Please don't associate this look with the "HDR" concept. HDR imaging is much bigger, much more powerful than that and it's the future of image-making. It takes the RAW philosophy and pushes it much further.

The examples Roberto Lequeux posted are of the overdone tonemapping kind, while the ones Digital Coast Films posted are of the subtler kind. It wouldn't be intelligent nor in line with RED's philosophy to build a camera that spat out pre-baked tone-mapped images. It would wreak havoc with linear vfx workflows, to name one problem.

You can consider Red's raw 12bit HDR data in respect to the final product, so when you have a Red shot that you can't fit satisfactorily into an 8 or 10bit space using only curves, if you want to use tone mapping don't do the double ISO export dance. That's just stupid. All your data is in the original 12 bits. Export a single linear 16 bit Tiff or OpenEXR sequence and tonemap that with your preferred algorithm. Or import the sequence into a compositing tool with a linear color space (Nuke, AE, etc) and dodge and burn, make selective tracked masks, custom grads and re-expose to taste.
As we talk more about HDR motion acquisition I think we have to start talking about the implementation of TMO's in our color grading apps and maybe rethink the use of LUTs in the pipeline.

On the speculation on RED's method for HDR motion, you can search the SIGGRAPH archives for HDR video projects (can't find a free link to the whitepapers now). Alternating exposure schemes usually apply optical-flow interpolation to simulate multiple simultaneous exposures from contiguous ones. It's an intensive calculation and far from perfect. It might result in artifacts around motion depending on the allegorithm's strengths. So I doubt a red brain implements that in-camera. In that case it would most probably spit out the raw alternating sequence and depend on the post workflow to get it right. But I don't know if that's Red's strategy. Arri has an interesting approach with their multiple gain path. I'll wait and see what Graeme and the other Red geniuses have in store for us. I trust that they will do something spectacular and that people who don't understand real world limitations will find reasons to whine anyways (reminds me of the silicon-native 5500K sensitivity issue).

 

[Pawel Achtel]

Well said, Daniel.

 

[paulherrin]

good thoughts, thanks man.

 

[Eki Halkka]

Thanks! - some of this is patent pending by us, but I seriously doubt Red is infringing on our patents and copyrights. If they are they are doing so completely unknowingly/by coincidence which is part of why I post this here.

When did you file the patent?

Just a heads up, your idea is very similar to the one i posted here, originally in November 2008...

http://reduser.net/forum/showthread.php?t=31683&page=3#25

...if your patent claim was filed after that, your patent may very well be void because an idea is not "original" after it's been discussed in the public.

 

[Lauri Kettunen]

But it is still a net win if the increase in gain overcomes the input noise (including quantization noise) of your A/D

John, what about if the noise of the amplifier/gain is lower than that of the AD-converision? In such case can't see immediately a situation resulting in a net win as you described. Since you are pretty well up to the technical matters I'm just curious if you know more about the subject.

 

[Spike Baumann]

I don't know if anyone mentioned this but this should allow you to use video to light 3D scenes using GI capable render engines. This would allow the scenes to change from inside to outside in one shot and be able correctly be illuminated. Architectural fly-throughs could benefit from this.

 

[Rick Burnett]

You are correct, any amplifier will always add noise. But it is still a net win if the increase in gain overcomes the input noise (including quantization noise) of your A/D. I believe that is the case with most DSLRs, at least up to some "reasonable" ISO. The latest DSLRs include extreme ISO settings which I think are more about marketing than about image quality.

I'm actually not talking about the amplifier noise but the CMOS sensor noise. My understanding is that the CMOS sensor, whether passive or active (and I'd imagine these are active) have noise issues. If you amplify the voltages coming off the sensor, all you are doing is amplifying the CMOS noise as well. You are right that in this method, you avoid digital amplification which would amplify not only quantization noise but any noise caused by the A/D circuit (like thermal as well).

But I reiterate, I don't understand why ARRI is doing what they are doing. From what they are implying is that their sensor is more dynamic than 14-bit so they are using two overlapping voltage windows to produce a 16-bit value. Maybe they want the freedom to gain those two stages separately, but I still think the proper approach would be to gain the whole sensor up enough to get the best balance between the noise of the CMOS sensor and the noise of the A/D converter, and just use a larger A/D. They effectively have 28 bits of A/D (although overlapping it appears) to make 16. Certainly more power required. Their method with the high gain side would provide a lowering of the noise inherent (but not quantization) to the A/D converter which could lead to differing noise across the image (the quantization error will still exist once you take the high gain digital values and put them back where they belong since the quantization problem is due to bit bin sizes of the digital representation.)

 

[Jeremy Hughes]

Wow, so much speculation! I think it's going to be great regardless of the method used. Can I throw out my lighting equipment yet?

 

[Jannard]

We won't do HDR the same as anyone else just a no one has developed a sensor with the noise characteristics of the M-X sensor.

Jim

 

[Digital Coast Films]

We won't do HDR the same as anyone else just as no one has developed a sensor with the noise characteristics of the M-X sensor.

Jim

Even more encouraging!

Would love to see this demonstrated at NAB