HDR Mode...

[Rick Burnett]

I guess that begs the question of what does 'baked in' mean in terms of HDR and what would the advantage be to baking in the extra dynamic range in camera? Does this mean the output actually has a larger dynamic range for each pixel at this point? This would be my desire. This is what I considered not baked in.

Until Red spills the beans on how they are achieving a higher dynamic range, it's hard to say either way really though. I based my speculation on the multi-read before reset approach, and the possibility of the blending of frames could be done in a multitude of ways depending on the actual dynamic range that is trying to be captured. In this case, I'd think you'd want to have the opportunity to blend differently based on content. For instance, a threshold based blend where you ignore content that is under exposed or overexposed respectively (simplisticly speaking).

But then again, HDR isn't a quantifiable target. Maybe these HDR techniques for video don't add a huge amount of extra range, but just a little extra. In these cases, I could see the possibility for a baked in approach where you get extra detail in your image. But that would imply tone mapping since you'd be reducing the HDR range down into a smaller dynamic range which seemed to be implied that is not the case.

So like I started with, what does baked in actually mean here?

 

[Gavin Greenwalt]

Good point. I would assume Baked-in means still a linear image but as a single half-float/float frame.

Based on Graeme's previous comments on tone mapping I think it's safe to say RED would never deliver a non-linear RAW image.

 

[Richard Goodwin]

I could be wrong, but couldn't the Arri Alev high/low gain path also be described as HDR?

http://www.arridigital.com/teaser/sensor

"The ALEV III sensor design provides 32 pairs of outputs. Each channel is divided into a high gain path (H) and a low gain path (L), resulting in 64 channels arriving at the 14 bit A/D converters. In the final images the lowlights are reconstructed from the high gain path and the highlights are reconstructed from the low gain path for an image containing meaningful information in all 16 bits."

I don't think this is suggesting HDR. What it implies to me is that they are diving up the image into highlights and lowlights. This allows for better processing of all the information. In this specific case they are saying they can produce higher quality 16bit images. This is because the A/D converter is only 14bits. So if they fed everything through there they would essentially "truncate" the 16bit image data. By dividing it they don't have to truncate it and it can be fully processed and the "re-assembled" afterwards.

I'm reading into this a bit because I'm not sure if they are saying 16bit come off the sensor or not.

If this were HDR that would imply that the sensor itself had higher dynamic range (because this description appears to be about the processing of only one exposure).

 

[David Rasberry]

As far as the final processed HDR image is concerned, it is compressing the available dynamic range of the original(s) into the narrower DR of a display format, whether electronic or print, in a way that maintains important mid tone contrast for low light and highlight regions.
I'm thinking they are going to do something like overlay two or more different FLUT™ adjustments of a single exposure to achieve the HDR look. Makes the most sense for motion shooting. I don't see them doing exposure bracketing during the actual shooting or necessarily needing to.
There is a night time Tokyo shot from Paul Leeming that I have been playing with in Redcine-X that would benefit from this kind of processing even though it is an older build 17 .r3d file and has about 3 stops less DR than M-X.
It is a walking shot of the Shibuya crossing. I can adjust for decent contrast and detail in the low light areas, but lights, windows and signage displays burn out, If I adjust for the highlights you get no shadow detail. If I flatten the gamma to get highlight and low light details the image loses contrast. If you could apply multiple FLUT™'s to the image it would maintain both contrast and detail throughout the range like an HDR image. All the data necessary to do this is in the single exposure raw file as it is. Add the extra 3 stops of DR from M-X and it could have the HDR look pretty easily.

 

[Stephen Aldridge]

I wonder if the wizards at Red have figured out a way to use ISO variably across the same image. Say, 100 ISO for the highlights, 400 for the mids and 800 or better for the shadows. With the match light shot and Offhollywood's amazing New York-at-night footage, we know that the new sensor can easily handle the higher iso with perfectly acceptable noise levels so if the camera/sensor could use this method, and if the operator could set his highlight ISO, mid ISO and shadow ISO, I think that would be pretty cool. Maybe someone has already suggested this? I haven't had time lately to read all the posts.

 

[Roberto Lequeux]

This thread is giving me a headache...

:beatdeadhorse5:

 

[paulherrin]

there always has to be semantics issues with hdri... one of these days we'll figure out how to communicate. i highly doubt anything will be "baked in" with this mode. it's hard to say what to expect. it could be as simple as an automation of using the latitude of the r3d to generate a 32-bit image, which would be helpful for some in certain situations, especially for visual effects (shooting a reflective ball for cg reflections and image-based lighting, among other things). the fact is, the tone-mapping that might be done is really not so different from the way we are already used to manipulating photography and film - it just gives us more room to totally screw it up, like some of the infamous tone-mapped images we've seen. and the manipulations we're doing are nothing new from the advent of computers, take a look at some of the masters work like Ansel Adams, very cool techniques for getting the most out of his negatives. it is true - with great power comes great responsibility. and just as much as it gives us the option of destroying an image, hopefully we will yield that power well. i'm excited for this feature however it comes, because i know RED will do an outstanding job and surprise us once again.

here's a cool example of adam's printing notations for what essentially could be called tone-mapping.

 

[David Battistella]

That's 'cause everyone knows the answer to stupid questions. Not many people know the answers to tough ones. Keep asking.

It seems like such a simple question. Why is the answer so politically charged?

I just don't get it sometimes.

 

[paulherrin]

david, just ask yourself what RED's policy on baked-in anything is. other than really good cookies baked-in the oven. Jim claims a clever approach. In my opinion, on camera gain processing, or otherwise baking in anything with this HDR mode other than a 32-bit image, is not very clever. of using the latitude of the sensor combined with that of the r3d's seemed clever to me until i realized i probably wasn't the first one to have that thought, but i still see it as a legitimate practice in some scenarios, especially so with the new sensors. but RED seems to be far more clever than i am when it comes to scientific and techno-logics, so i won't be surprised when i'm surprised. the point is, they're not going to give us something worse than what we can already get. we can already generate pseudo-HDR's from the existing cameras without introducing too much noise or quantization, it just takes a little custom workflow - and a baked in or automatically tone-mapped images would be far worse. so i wouldn't worry.

 

[Pawel Achtel]

Well here is another theory

If the sensor can have different setting that enable it to see in darker part of the image, and it also have settings to see the brighter part AND you can switch settings fast enough. Then it is "just" a matter of capturing at both hi and low ends at the same time, and use whatever blending method you prefer. Maybe even every other pixel can have different settings, and use a clever de-bayer to process it.

..the fun begins when you try to shoot in 3D with one sensor AND in HDR

Unfortunately, because your assumption between the "If" and "Then" is, to my knowledge, false, the theory does not seem to be very promising

 

[Zach Gray]

Folks kept talking about motion blur artifacts, and I wanted to see what they might look like. Judge for yourself.

The two most realistic ideas seemed to be combining images shot at high frame rates and a rapid read with no reset.

Maya/MentalRay allows you to specify the shutter angle, and also create a motion offset where only a portion of the time of the frame is considered in the calculation. I didn't render the images at different exposures in this experiment, but that would be a given with changing shutter speeds.

Essentially, I could recreate the ideas suggested here with accurate blur. I set up a scene with a 50mm lens with a 35mm filmback at 10cm away from a 2cm ball. I did experiments at 100cm/sec and 10cm/sec.

These images are averaged with photoshop. It might be cool to see what Nuke/Furnace could do with a sequence using motion estimation.

 

[Gavin Greenwalt]

Zach did you do the blending in linear space or sRGB?

 

[JanneJansson]

Unfortunately, because your assumption between the "If" and "Then" is, to my knowledge, false, the theory does not seem to be very promising

That's why it's a "theory" :emote_hippie:

Sorry I'm a sucker for thech-theory
To start talking about HDR, implies you are doing something extra other then using the sensor traditionally. That extra can be exposing multiple frames at high speed and combining to new frame. Can also be using all those lovely pixels i a new way. Lowering resolution and you may be able to expose multiple exposures at lower resolution at the same time on the sensor surface. In the same way a bayer pattern for color, you could have groups/lines on the sensor just for hi or low light levels. Time sharing, surface sharing or new way of process raw.

.. but we see soon what the tech wiz at red have done

Best

 

[Eki Halkka]

Everyone has a guess...

You're implementing this?

http://reduser.net/forum/showthread.php?t=31683&page=3#25

;-)

 

[Pawel Achtel]

You're implementing this?

http://reduser.net/forum/showthread.php?t=31683&page=3#25

;-)

This is my guess too

 

[Zach Gray]

Zach did you do the blending in linear space or sRGB?

I used the defaults in mr. I didn't use a lens shader and rendered to a 8bit frame buffer. I did this at home, so I just layered the images in photoshop. I'm going to assume sRGB. When I render16 bit exr images at work, I interpret them as sRGB in nuke. I've read your previous posts on the benefits of a completely linear workflow, but haven't implemented that in rendering.

It would be interesting to play with this more and actually throw exposure into the mix.

 

[Steven Caesare]

I would have to imagine it's baked in. If there were no side-effect then RED would be doing it for their 'normal' image mode. If there are side effects then some aspect of the process will be baked in.

So it's pure speculation but no matter the approach logically speaking I think we would have to conclude it's baked in.

Or it requires a specific step in the processing chain (like debayer).

I for one would be surprised if it were baked-in in the the sense that it irreversably altered the raw sensor data. I suspect it's more along the lines of taking the additional frames, specific metadata, etc...

In that sense it may not be the exact same as non-HDR RAW data (ie. if mutiple frames are read without an itervening reset), I suppose you could say it might be a baked-in change however.

-sc

 

[paulherrin]

I used the defaults in mr. I didn't use a lens shader and rendered to a 8bit frame buffer. I did this at home, so I just layered the images in photoshop. I'm going to assume sRGB. When I render16 bit exr images at work, I interpret them as sRGB in nuke. I've read your previous posts on the benefits of a completely linear workflow, but haven't implemented that in rendering.

It would be interesting to play with this more and actually throw exposure into the mix.

yeah, thanks for that testing zach, exposure may throw new light on them. also, motion will be absolutely critical. the movement of the camera, vs. the movement of a subject in a locked down shot will be quite different.

 

[Justin K Phillips]

This is my guess too

Mine, as well. It's the only one that really makes sense.

 

[Rick Burnett]

You're implementing this?

http://reduser.net/forum/showthread.php?t=31683&page=3#25

;-)

Have you done some thinking about what the advantage or disadvantage of using successive frames as you say, or if you disable the reset between successful frames? The nice thing about using successive frames that build on each other, let's say three frames, is that the final frame contains content of all three, whereas in your method, they could each have different content if something was moving fast enough. The non-reset would have more motion blur, which could hide the ghosting or just double/triple/whatever images.

The more I thought about this, the more I realized I have the equipment now that I could test this. Unfortunately, I do not own a R1 camera, so someone with a R1 could do this test better (what is the max framerate?) but let me explain and someone else can try this (or I can try to do this with my 7D although it isn't going to work like I'd like).

1) Shoot 24p video, setup a scene that will either over expose or under expose, and I'd shoot it in both scenarios for reference.

2) Shoot at 3 times that if possible, or if not, at least 60p

This will be an iterative process but what you are trying to do is figure out at what settings 1 images is pretty dark, 2 successive images added together are somewhere in the middle and 3 added are over exposed. I'd do this with frame grabs in Photoshop since you can use the HDR stuff built in by fooling it with three frames (1, 1+2, 1+2+3). Or, if you have another method which allows you to bring the frames together in a different way.

Obviously the faster the framerate you can shoot, the less motion, but also, the smaller total sum of range as well if that makes sense, but if you have something that moves, put it in front of the camera. (And you can use shutter speed to help as well if you cannot get a high enough frame rate).

Once you get this setup decently, you will have three frames of data that you can bring together to simulate Eki's method, or other methods. How to blend these, I'll have to experiment with. But, the original video you shot can be used as a comparison to the results you get.

Remember, the sensor is just a summation of photons hitting it, so taking successive frames and adding them is like ignoring the reset. Since I cannot get RAW out of my camera, it makes this test a bit hard.

I'll see what I come up with. I think the big problem is I might need to write some scripts once I figure out what I want to do that can take a video source and generate something based on three frame groups (or even 2 frame or whatever). Not sure which app I want to do that in.

Edit: Actually, might make sense to just use two frames for a test just to simplify the whole process.