HDRx optimization

[Michael Tiemann]

Joshua, your two posts really illustrate well the caveats about using HDRx "just because". In the first post, where you layer together the A- and X-frames to create a super-wide dynamic range image, you realize that beyond a certain point, more dynamic range leads to more, not fewer problems in reducing the image back to something that looks visually credible. In the second post, where you merely grade a reasonably over-exposed or a reasonably under-exposed image, you get very sane results (because, honestly, there's a lot of intrinsic dynamic range in a reasonably exposed R3D image). Thus, in these cases, HDRx is not a tool that's really helping you with your underlying problems.

Here's a posting from a thread where I did have an identified problem that HDRx helped me address: overly bright LEDs that I wanted to tame while still having a concept of normal ambient light and shadows. TL;DR: HDRx can be a great tool if you have a really specific idea of what aspect of the image you want to fix via less/more exposure and/or longer/shorter shutter speed. (You obviously don't get one with the other, but the limiting factor of one may dictate the limiting factor you can correct with the other.)

About your luma key: I would have turned things around and used custom RGB->Luma equation to pick the brightest and bluest pixels from the X-frame and then blended them in (with a fairly long "soft" value in Resolve) over top the A frame. This would bring your sky down and then let you subsequently grade your (mostly not blue) shadows. The post I linked to above gives an example as to why one should not trust the default RGB->Monochrome equation, especially when your area of interest is not completely desaturated.

 

[Marc Wielage]

I've been messing around with this. It's frustrating to see all the shadows exposed well in the A Frame and the highlights exposed well in the X Frame and then blend them together and it doesn't look much different from exposing in the middle. So I took each stream into After Effects, duplicated the A Frame, turned the top layer into a luma matte for the bottom layer, and set it to "lighten", so only the darkest areas of the overexposed A Frame would affect the underexposed X Frame. It came out looking pretty flat indeed. So then I brought it into Premiere and graded it (in Lumetri just maxed the contrast and tweaked the curves to deepen the shadows), and it does in fact look a little better than if I take just the X Frame and grade it (starting with RedLogFilm). Better enough that it's worth the effort? Probably not. But this technique might be useful in other cases.

We basically do that with windows and masks and all that stuff in color correction. This technique goes back 90 years to Ansel Adams "burning and dodging" in the darkroom: brightening the areas that need an increase in exposure, then darkening areas like skies that need less exposure. The trick is doing it while movement is going on.

For the shots in your example, I think just taking the down-the-middle exposure could still yield decent results if you put a couple of windows on the monument and color corrected that separately from the background. It is possible to save the sky and the background if you work at it. Modern cameras really do have a lot of latitude, but I don't dispute the argument from Michael above that if you have blown-out LED's or something like that in the frame, it's rough to save them. In a perfect world, these need some ND's on the set, but I get that this isn't always convenient or possible.

I just had a situation in the last week with some Red sequences shot in a florescent-lit hallway, and I had to do quite a bit of work to avoid letting the ceiling lights go nuclear. It took a degree of work to figure out a way to walk the tightrope between overexposure and yet still give the actors below enough light so they looked reasonably natural. But I didn't need HDRx, and I was probably able to knock the highlights down roughly 2 stops without affecting the people. It just takes work.

 

[Christoffer Glans]

All I can say is I've worked on four or five projects that tried to employ HDRx, and the results were mixed at best. In at least half the cases, we were forced to jettison the additional exposure because of motion artifacts. It's often much wiser to go with a simpler, cleaner post workflow -- partly to save time and money, but partly to get a more organic image in the end.

Trying to justify the use of technology for bad lighting, or (understandably) for limited experience or a severe lack of budget, is the biggest folly here. Trust me: just light it right the first time, and you won't have to use HDRx. Take more time and money on the set and fill more.

You seem to fail to notice that by doing it in the way I did... you don't get the motion artifacts you mention here. Also, there will be moments when you don't have either time or money to light it in the way it's supposed to be, I would say that an excellent notion of limited experience is the belief that every project will go smooth and according to plan. Having this technique up your sleeve could save a shot if properly done. With more experience people will realize just how chaotic a film shoot or any shoot can be and I like to have control over my variables, instead of just acknowledging how things "should be done".

We basically do that with windows and masks and all that stuff in color correction. This technique goes back 90 years to Ansel Adams "burning and dodging" in the darkroom: brightening the areas that need an increase in exposure, then darkening areas like skies that need less exposure. The trick is doing it while movement is going on.

For the shots in your example, I think just taking the down-the-middle exposure could still yield decent results if you put a couple of windows on the monument and color corrected that separately from the background. It is possible to save the sky and the background if you work at it. Modern cameras really do have a lot of latitude, but I don't dispute the argument from Michael above that if you have blown-out LED's or something like that in the frame, it's rough to save them. In a perfect world, these need some ND's on the set, but I get that this isn't always convenient or possible.

I just had a situation in the last week with some Red sequences shot in a florescent-lit hallway, and I had to do quite a bit of work to avoid letting the ceiling lights go nuclear. It took a degree of work to figure out a way to walk the tightrope between overexposure and yet still give the actors below enough light so they looked reasonably natural. But I didn't need HDRx, and I was probably able to knock the highlights down roughly 2 stops without affecting the people. It just takes work.

So, your point being that you didn't need HDRx for that shot, but me hiring you by the hour to grade something I would have to pay for a lot of extra work that could have been handled better with HDRx on a shot that might have been impossible or taken too long on set to prepare? There's a difference between handling things as they've always have and finding new better ways to do things. There's definitely shots that are impossible to do on set, like stopping down a sky with ND grad while having an actor not being affected by that grad if standing in front of the sky. Yes, you could window that actor in post, but then you also push up noise because that area of the image is darker.

My point is still that there will always be situations when you can't do it "as it's always been done", since some shots are impossible in the old ways. Using my 50fps HDRx technique I could get those nice looking dusk and dawn shots where you traditionally need to push up a lot of the environment in order to keep the nice looking sky. With HDRx I could get a nice noise free exposure of the environments while still getting perfect exposure in the hot spots around the setting sun, while being able to do it hand held, with actors moving all over the place. If that's not a powerful exposure-tool that should be explored I don't know what is.

 

[Marc Wielage]

My point is still that there will always be situations when you can't do it "as it's always been done", since some shots are impossible in the old ways.

Did you actually try it with color-correction? You'd be amazed how often a skilled colorist can bring out more from the image, as long as the highlights aren't totally burned-out radioactive.

 

[Christoffer Glans]

Did you actually try it with color-correction? You'd be amazed how often a skilled colorist can bring out more from the image, as long as the highlights aren't totally burned-out radioactive.

Yes, I've been working as a colorist on and off for the last 15 years so I've done exactly what you describe. But even if I can get back highlights in most cases I cannot change simple facts about the DR that exists in our cameras; if there are images shot with all highlights protected and everything else gets crushed blacks you have an impossible exposure to fix in grading. You could try and lift everything but you get noise or not even anything at all. If you then try and get an exposure that's protecting as much as possible while still having information in the dark areas, you will still not get all the highlights looking nice since parts of them are burning out. In those cases you have to roll them off nicer and you probably smooth the highlights into a burnt out look even if it rolls off.

Shooting 6 stops HDRx without the usual artifacts, it will protect pretty much everything. You will not have overly noisy dark and midtones, you will have every light source and highlight with workable details and if it still blows out you have a lot of range to do a roll off with more control over it.
The thing I notices is that I can now grade an image to perfectly mimic how my eyes see. If I sit in a room lit with only a window, I can clearly see everything inside and still see everything outside. There's nothing burning out anywhere.

There's also another feature to this and that is that whenever you shoot a scene where a character goes into a building and out again (and you can't light it because those lights are gonna appear in the shot) you traditionally always need to change aperture when doing it. This affects not only how the scene looks when entering the building or exiting (crushed darks going into range and vice versa) but also how the DOF is inside compared to outside. Using HDRx I could do such a shot without changing anything other than focus and still have everything I need when grading. I could have a long steadycam shot that goes in and out of darkness and not being worried about aperture changes or DOF changes. It's especially handy if you can't change the aperture nicely, like on still lenses or if the remote focus is only single gear.

There will still be shots that need more DR than what any camera can produce today and increasing that when you need it definetely has it's place. We have just been limited by the artifacts, but shooting 50 or 48fps would erase that problem. Of course there's no point in shooting a whole movie this way since the storage space needs to be almost four times more.

 

[Mike Nagel]

Chris,

have u tried Nuke or Mamba for these few shots that need it ?

the HDRx shots (unfortunately) usually need to be composited in an app that gives u more controls over various parameters used for blending the two tracks, ergo a real comp app...

- M

 

[Christoffer Glans]

Chris,

have u tried Nuke or Mamba for these few shots that need it ?

the HDRx shots (unfortunately) usually need to be composited in an app that gives u more controls over various parameters used for blending the two tracks, ergo a real comp app...

- M

Haven't tried HDRx compositions in Nuke or Mamba, but I heard Mamba is a very good option for it. But if I do my 50fps workflow I usually only need to compose it as a simple blend using the RAW settings, adding motion blur erase the usually bad quality compared to magic motion.

 

[Marc Wielage]

There's also another feature to this and that is that whenever you shoot a scene where a character goes into a building and out again (and you can't light it because those lights are gonna appear in the shot) you traditionally always need to change aperture when doing it.

Yes, I've dealt with this problem for many years. I've had a few arguments with camera operators and DPs who insisted that they needed to change the F-stop, and I'd argue that when they do that, I wind up having at least three different corrections to ramp in and out: the initial indoor correction; the change to the outdoor as the aperture jerks open; and the final outdoor correction. It's the changes that are non-liner and unpredictable, and I usually wind up spending more time just trying to cover those up. I can recall many 16mm projects in the 1980s where we really had to struggle with this.

Nowadays, if you have 6-7 stops of usable range in a camera, then usually -- but not always -- we can work with a "middle ground" correction and just bring up the interior and bring down the exterior. Generally. There are always cases where the interior is underlit and the exterior is blown out, and for that, then we can use an F-stop change if it's done very smoothly.
I think you're trying to justify the use of camera technology where it's actually very simple to work without it. I've done countless Sony and Alexa projects in the last 4-5 years where we had no HDRx, and yet somehow we limped by and everything looked fine. We routinely have to keyframe hundreds, maybe thousands of shots for exposure, and we often have to use tracking windows to control the exposure and adjust where and how the images within the frame are handled. I would point to Steve Scott's work on Birdman (which won an Oscar for DP Chivo Lubezki) as an example of an Alexa film that had vast exposure changes in continuous shots without the need for HDRx. I think in that case, the dark scenes were more problematic than the light ones, plus they had the problem of how and where to hide the lights.

 

[Christoffer Glans]

Yes, I've dealt with this problem for many years. I've had a few arguments with camera operators and DPs who insisted that they needed to change the F-stop, and I'd argue that when they do that, I wind up having at least three different corrections to ramp in and out: the initial indoor correction; the change to the outdoor as the aperture jerks open; and the final outdoor correction. It's the changes that are non-liner and unpredictable, and I usually wind up spending more time just trying to cover those up. I can recall many 16mm projects in the 1980s where we really had to struggle with this.

Nowadays, if you have 6-7 stops of usable range in a camera, then usually -- but not always -- we can work with a "middle ground" correction and just bring up the interior and bring down the exterior. Generally. There are always cases where the interior is underlit and the exterior is blown out, and for that, then we can use an F-stop change if it's done very smoothly.
I think you're trying to justify the use of camera technology where it's actually very simple to work without it. I've done countless Sony and Alexa projects in the last 4-5 years where we had no HDRx, and yet somehow we limped by and everything looked fine. We routinely have to keyframe hundreds, maybe thousands of shots for exposure, and we often have to use tracking windows to control the exposure and adjust where and how the images within the frame are handled. I would point to Steve Scott's work on Birdman (which won an Oscar for DP Chivo Lubezki) as an example of an Alexa film that had vast exposure changes in continuous shots without the need for HDRx. I think in that case, the dark scenes were more problematic than the light ones, plus they had the problem of how and where to hide the lights.

I still don't see the reason for not using HDRx on shots that demand extreme DR. Yes, you can do all this, but smooth aperture changes need to hit the mark exactly and maybe one take the acting was good but the focus puller didn't hit the aperture-mark. As a director I want as less risk variables as possible, especially if it could make or break a shot and render it useless even when the acting was spot on. You can always do another take on pure technical terms, but for acting, sometimes you only get gold once. I do not want technology to get in the way of that, so using HDRx definetely makes certain shots less risky.

Thing is that it's still true that just because something has been done for years in a certain way doesn't mean new tech shouldn't be used. It's like if we went back to the dawn of DI and said that Oh Brother Where Art Thou was totally unnecessary to do with DI since labs have been balancing colors for years. We don't need DI because we've done it in a certain way for 80+ years.

Of course I know that you can get most shots without HDRx, but I'm talking about those extreme DR situations where you either couldn't or need other things to help you out, like changing aperture or using ND grads. Those tools have an impact on the image in other ways, HDRx does not. Doing it in a traditional way also, as you remarked upon, force you to do a lot more with grading, which for me would cost money if I hire someone else to do it. Time and money that I could put into something else without affecting the primary goal of the grading session, i.e the creative impact. It's also important to note that pushing images that were shot underexposed to keep highlights always increase the noise. -There are numerous setbacks with doing things as they've always been done. It doesn't matter if it "worked great", it cost time, money and resources, forces you to do it in a certain way, maybe even block actors based on it which is insane. If HDRx helps to reduce such restrains on certain shots, then it doesn't matter how things have been done, it's still a better way. If you do all that extra grading work for free then fine, but I doubt you don't charge for all those thousands of shots you need to keyframe and put windows on?

Outside of all this, you also get a kind of shot that you can't get without enormous VFX work and rotoscoping. People remarked on how good the cinematography is on Better Call Saul and part of that is the "HDR" shots where you see the outside and inside of a room in perfect exposure. Even if they didn't use HDRx, that kind of imagery is more so possible with HDRx, so you might not think it's a good tool for helping grades, but for me as a director it's a great tool for realizing my vision.

 

[Mike Nagel]

the "problem" is that HDRx was a feature that excited many, but in actual practice was/is not much usable (considering the gain vs. extra expense) with the standard tools available, other than going full comp - which again is expensive and may better be reserved for those few, select special shots...

but, I fully agree with all of your points and would love to have this as a full on working solution that is convenient and pain-free to implement... not holding my breath though, this feature has been around for 5 years, highly doubt u'll see dev progress on this...

 

[Christoffer Glans]

the "problem" is that HDRx was a feature that excited many, but in actual practice was/is not much usable (considering the gain vs. extra expense) with the standard tools available, other than going full comp - which again is expensive and may better be reserved for those few, select special shots...

but, I fully agree with all of your points and would love to have this as a full on working solution that is convenient and pain-free to implement... not holding my breath though, this feature has been around for 5 years, highly doubt u'll see dev progress on this...

Have you tried the 50 fps technique I described earlier?

 

[Mike Nagel]

no, only did full comp in Nuke on some shots years ago... I agree w/ Marc that a lot of times the DR especially of Dragon is enough (especially when it's shot correctly), not that I'm not agreeing this is not super useful...

but the nature of the beast is also that this blows up the mags like crazy, especially at 50fps, mags that need to be off-loaded, verified, confirmed etc... and I do a lot of takes, so there'd be an enormous amount of extra data to deal with in 99% of the shots zero benefits, and the few special shots can be dealt with in real comp with full control...

question is also how does your 50fps technique look besides getting rid of the artifacts ? does it look good ? not from a general QA point, but does it look like a real shot or fake ?

 

[Christoffer Glans]

no, only did full comp in Nuke on some shots years ago... I agree w/ Marc that a lot of times the DR especially of Dragon is enough (especially when it's shot correctly), not that I'm not agreeing this is not super useful...

but the nature of the beast is also that this blows up the mags like crazy, especially at 50fps, mags that need to be off-loaded, verified, confirmed etc... and I do a lot of takes, so there'd be an enormous amount of extra data to deal with in 99% of the shots zero benefits, and the few special shots can be dealt with in real comp with full control...

question is also how does your 50fps technique look besides getting rid of the artifacts ? does it look good ? not from a general QA point, but does it look like a real shot or fake ?

Of course, I mentioned it as well that it's not something you would do on every shot, but for those that need a super high DR, HDRx is something very unexplored that I think is a bit unused as a technique.
You have to apply motion blur, but even without it you get rid of most of the ghosting. These were shot at 6 stops HDRx, which usually gives the most extreme ghosting.

Passwords: HDR

 

[Marc Wielage]

the "problem" is that HDRx was a feature that excited many, but in actual practice was/is not much usable (considering the gain vs. extra expense) with the standard tools available, other than going full comp - which again is expensive and may better be reserved for those few, select special shots...

That's basically my experience as well. 90% of it to me just boils down to use more fill light in the darker part of the shot. It's doable, not necessarily expensive, and not necessarily time-consuming.

I just had a scene like this in a restoration of a 1976 B&W docudrama, and the director told me what we wound up with was the best he had ever seen that shot look. So a lot has to do with the fact that our post tools today are often better than what they had to work with in the film days. What helps with digital right now is that the basic dynamic range of the cameras is far better now than it was (say) 7-8 years ago.

BTW, I was reminded by a friend of mine that some of the transitions on Birdman were carefully executed composites, where they optimized the interior exposure, optimized the exterior exposure, and then as the cameraman walked outside, a precise split-screen happened that joined the two shots together. So that's yet another way of blending two different exposures together: as two different shots. There's a lot of different ways to approach this problem.

 

[Christoffer Glans]

There's a lot of different ways to approach this problem.

...but non are as simple and fast as just shooting without thinking about it on set and just have enough DR for both interior and exterior in post. I still don't see any really good arguments against using HDRx if you manage to get non-ghosting material out of it. If 90% of a film shoot consist of regular shots that are manageable within the base-DR and 10% are special high DR shots with 50 fps HDRx, the cost of those shots would be far less than hiring multi gear remote focus, ND grads, together with comp, extra grade and so on. Thinking purely about the cost, the cost of some 10% extra hard drive space for storing the heavy HDRx files is far less then any other technique for getting these kinds of images.

 

[Michael Tiemann]

Christoffer,

I think you are missing the point of Marc's wisdom. The lesson from Birdman is not that a split-screen comp is necessarily better than an HDRx approach, but that proper assessment of challenges, tools, and solutions can lead to great results. I happen to agree with you that HDRx is a useful tool in the toolbox, as I have shown with my own sample videos (where I judge 2-stops of HDRx to give the best answer in this case):

And it's prone to artifacts, as this 6-stop HDRx version shows:

But the larger point is that one should evaluate the full variety of approaches that are available for a given situation and choose the best, rather than picking a favorite because it's a favorite. Saying "it's the best solution when you don't have the time/cannot be bothered to prepare differently" is low on the list of good reasons to use a given approach. I agree with you that there's powerful magic wrapped up inside what the HDRx capability can do, but that magic is best used when it's part of a coherent plan, not as a way to better enable post hoc solutions that were indeed only necessary because of a lack of a better plan from the beginning.

 

[Mike Nagel]

The only thing that matters is the final IQ of the shot.

I agree with Chris that HDRx is by far the best solution (for these specialty shots) if the final end product looks at least as good and/or improves upon the other options.

The main problem has been all along though that nobody got consistent, good results using HDRx so nobody bothered anymore...

So question is whether the 50fps, high motion blur workflow can deliver footage that is on-par IQ wise with the other approaches, and more importantly can be used on all kinds of shots, not just locked-off stuff...

I'll test this out when I get a chance...

 

[Christoffer Glans]

Christoffer,

I think you are missing the point of Marc's wisdom. The lesson from Birdman is not that a split-screen comp is necessarily better than an HDRx approach, but that proper assessment of challenges, tools, and solutions can lead to great results. I happen to agree with you that HDRx is a useful tool in the toolbox, as I have shown with my own sample videos (where I judge 2-stops of HDRx to give the best answer in this case)

Absolutely and I've never disputed any of the traditional ways to do such shots as I've been doing those things myself. A music video I worked on had blown out windows and it was one of the first shoots on the Epic Dragon. I was stunned by the amount of DR you got and I split up that into two luma tracks, pulling back the outside into a visible range. But it's not even close to what I can do with a 6 stops HDRx. Did you look at the videos I provided above, that's 6 stops of HDRx without ghosting.

The only thing that matters is the final IQ of the shot.

I agree with Chris that HDRx is by far the best solution (for these specialty shots) if the final end product looks at least as good and/or improves upon the other options.

The main problem has been all along though that nobody got consistent, good results using HDRx so nobody bothered anymore...

So question is whether the 50fps, high motion blur workflow can deliver footage that is on-par IQ wise with the other approaches, and more importantly can be used on all kinds of shots, not just locked-off stuff...

I'll test this out when I get a chance...

Yeah, but just as any special technique in filmmaking it doesn't work out of the box. I think that people expected the magic of HDRx without putting any thought into it, that's why people turned it on, saw that it looked crap and turned it off. However, I really find using 50 fps on it and almost completely remove every bad artifact on it really changed the game for it. Yes, it's post pixel blur, but you really have to have extreme motion before that post-blur breaks down. You could also try putting a 360 degree motion blur on the 50 fps before converting it down to 25fps. That way the samples for motion blur is increased as well, helping higher motion, although it takes longer to render.

I'm still waiting for my delayed WC battery mount so I haven't really been able to go out and do proper HDRx testing. Right now all I got are those clips I posted earlier. But if you look at the movement of the hand and when panning the camera, the post motion blur and the absence of ghosting artifacts really makes a strong case for it actually working.