"HDR MODE" - Possible on Red?

[Nick Brian Walters]

I don't quite understand how are you suggesting that one would add these two photos together for luminance gains?



Also...
Then if the object is moving, say it's a white dog running across a black back ground. The dog is blown out by one stop using conventional frame rates and exposure.

In the second frame the dog will occupy a different space and the edge pixels will not line up to be matched together for luminance gains. This includes any edge. The collar, the eyes any color or luminance shift.

Again I say (from my perspective and with respect for yours) the only proper way to extend the DR would be simultaneous exposures from binned photosites or split beams.

I havn't played with it yet, but this program : http://www.mediachance.com/hdri/index.html claims to be able to automatically pin cushion multiple shots to make them match, which could possibly assist in footage that doesn't quite match.

 

[J. Eric Camp]

Still, then Pin Warping could easily create unwanted motion artifacts or uneven motion.

That software is designed for stills to compensate for shaky hands not motion. Either way to get HDRish gains the exposures also need to be bracketed.

It's all very intriguing and someone will develop some crazy software to shut me up but in the mean time I am still a propionate of two simultaneous exposures. Which I will be happy to test soon.

I welcome the discussion.

 

[Eki Halkka]

I don't quite understand how are you suggesting that one would add these two almost identical photos together for luminance gains?

I thought i explained very simply ;-)

Then if the object is moving, say it's a white dog running across a black back ground. The dog is blown out by one stop using conventional frame rates and exposure.

In the second frame the dog will occupy a different space and the edge pixels will not line up to be matched together for luminance gains. This includes any edge. The collar, the eyes any color or luminance shift.

No.

the result of combining two frames with my method is basically the exact same as doing one normal speed exposure, except clipping happens one stop later. Try to read my explanation with the cool ascii graphics once more...

Another way of thinking of it is to forget the actual technique, and think of it this way:

1. You start exposing a traditional 24 fps 1/48 shutter speed frame.
2. half way through the exposure you save what you have so far
3. you continue exposing the frame until you have exposed the frame for whole 1/48 sec.
4. You add together the first and second halfs of the exposure time, using floating point math that allows you to go over 100% with the values.

What you end up between 0% (black) and 100% (traditional white) is the exact same as just having one single exposure, motion blur characteristics included. The result of combining two consecutive 1/96 second exposures is the same as one 1/48 second exposure. It doesn't matter if the exposure is accumulated in one go, or in two parts.

But, as both half frames are underexposed by a stop (while still containing values from 0% to 100%), your combined highlights can actually go to 200% (superwhite), in other words, they clip one stop later, you get a stop more DR.

Again I say (from my perspective and with respect for yours) the only proper way to extend the DR would be simultaneous exposures from binned photosites or split beams.

Nope, combining multiple exposures works too.

Unfortunately, i don't have a Red One of my own, so i have tested the method only with a regular video camera - it more or less worked as planned, but due to gamma applied in-camera, and the lack of ability to shoot at 96 fps the results were slightly ambiguous... i was convinced though ;-)

Now, if one of you Red owners could do me a favor, and shoot a short clip (say, a second or so) with 24 fps and 1/48 (180 degree) shutter, and another second long clip with 96 fps and 1/96 (360 degree) shutter, i'd be very happy to demonstrate my method in practice. In order for it to work properly, the two images that are combined should be in linear color space, which means i need the original RAW files.

Oh, to show the dynamic range actually increases, the scene that's shot should have a lot of contrast, and the f-stop chosen so that some parts of the image overexpose in the 24 fps and 1/48 clip. Other than changing the shutter speed and frame rate, there should be no other changes between the two test shots.

Oh number two, the clips should have some motion, to demonstrate the motion blur characteristics.

 

[J. Eric Camp]

I thought i explained very simply ;-)



No.

the result of combining two frames with my method is basically the exact same as doing one normal speed exposure, except clipping happens one stop later. Try to read my explanation with the cool ascii graphics once more...

I understand the words you use and do not need to read them again.

Another way of thinking of it is to forget the actual technique, and think of it this way:

1. You start exposing a traditional 24 fps 1/48 shutter speed frame.
2. half way through the exposure you save what you have so far
3. you continue exposing the frame until you have exposed the frame for whole 1/48 sec.

But this is not what is happening. You describe adding two separate 1/96 frames together in lieu of a 1/48 frame. I know your method is to shoot 96 frames a second with a 1/96 shutter. Four frames will happen inside of the same temporal space that 1 will happen with the normal 24fps with 1/48th.
So, with the 360˚ shutter you suggesting that since the photosite will simple break it's exposure time between the first two frames that motion blur will be the same as the 1/48th frame when the frames are then "added together" via floating point math.

The non specific part, the part I don't see clearly is:

4. You add together the first and second halfs of the exposure time, using floating point math that allows you to go over 100% with the values.

I understand what floating point is.

I also understand what gravity is and to me this statement is like,
"You jump out of a ninth story window and make sure you land so you don't break anything."

I will say the idea is clever... I just have not seen the magic of adding two very similar frames together that will interpolate the motion blur, reduce noise and add DR.

Clever as it may be this technique with the R1 would act like this on set:

We are now looking at an image with a 1 stop reduction on set.
We would have with no in camera audio.
We are forced into 2k mode, with it's crop factor and less over sampling.
All play back would be slow motion.
Our ramping and shutter capabilities are extremely limited to the point of being non-existent.

With all this being said...

I will shoot your test and provide you with raw files.

I also want to compare a over sampled 4k image to the 2k uber-image.

Past that... my suggestion for a technique allows for the full functionality of the camera with the potential for higher DR gains, we are just left with a huge camera rig.

Both ways have merits and flaws, I am happy to help you "prove" yours.

 

[Eki Halkka]

I understand what floating point is.

I also understand what gravity is and to me this statement is like,
"You jump out of a ninth story window and make sure you land so you don't break anything."

Just don't forget to tie on the bungee cord and you'll be fine ;-)

I will say the idea is clever... I just have not seen the magic of adding two very similar frames together that will interpolate the motion blur, reduce noise and add DR.

Using two exposures can either reduce noise (when they're exposed properly then mixed together) or add DR (when they're underexposed by a stop then added together), but not both at the same time...

Clever as it may be this technique with the R1 would act like this on set:

Yep, clumsy as hell - the only thing that could actually be remedied easily, is the monitoring: simply setting the camera ISO to twice the actual target ISO will bring back the lost stop. The post would pretty much be a nightmare too with R1, so in vast majority of cases, shooting 96 isn't anywhere near practical.

I will shoot your test and provide you with raw files.

I also want to compare a over sampled 4k image to the 2k uber-image.

Very cool, thanks!! I'm pretty sure supersampled 4K will look better than 2K using my method, even if you need to underexpose 4K by a stop to get the same highlight protection.

But still, it will be pretty interesting to try the technique out with proper test footage.

Both ways have merits and flaws, I am happy to help you "prove" yours.

Well, let's say "test" at this point - i'm pretty confident it will work, but it's always possible i've missed some glaring technical hole somewhere...

 

[Nick Brian Walters]

I do have to point out one thing, which has been shown, explained, and worked with in the past with stereo photography. The fact that glare from a different exposure can show up on another exposure when it's on the same place your exposing......by building extra stuff into a one chip solution I can see major problems happening. I myself am going to be trying a three chip solution, i.e. the HVX-200, where each CCD will be split in half where the left side gets one exposure, and the right side gets another, I can forsee problems here, and I think they could be even bigger when incorporated closely together.

All this said I have to agree with some statements here that the only "easy" solution is multiple exposures at the exact same time on multiple sensors. I think it's great to brainstorm the possibilities, but if anyone wants to do this NOW...the solutions are:

1. Synching two cameras closely paired, both facing as much as possible straight forward, then matching with software in post and slightly cropping.

2. Synching two cameras and through the use of a beam splitter matching their angle as much as possible....the steps would still be the same as our first option, but less cropping would happen.

3. Synching two cameras and through the use of a beam splitter sharing one lens assembly....this would be more difficult, but could possibly be cheaper in the long run if perfected and sold commercially.....I could see there still being a need to use alot of the steps from 1, but the cropping would be extremely minimal.

To be quite honest I think that option 2 is the best....FOR NOW..There are allready 3d rigs commercially available that use the beam splitter setup, which could be used for aligning two cameras instead of offsetting them, you give one a bit more ND then the other, and you're ready to go....I'm oversimplifying, cause of course test shots would need to be done, processed, inspected, etc.....but this setup is possible with professional results.

Now if you're just playing around and experimenting like I am you can try option 1, and just put two camera's together then align, crop, etc in post...and try to do something cool.....heck get together with someone else with your exact camera(whatever it is you operate), and play around...even if it's a system that can't synch, it's still good to socialize and experiment with others

I think higher Dynamic Range IS the future.....the Panasonic HPX-170 already incorporates some abilities to have more dynamic range....but it's easy with their system since they are using three CCD's....this option is also on their more expensive larger chip cameras. Will Red go this way? Only the future can tell......

 

[Noah Yuan-Vogel]

@eki

While I understand there is some argument about the definition of HDR, im not sure I would call combining two images with with equal exposure to extend DR up to 1stop HDR. Nor do I feel that it is likely to be worth the hassle unless it is implemented completely transparently or is a positive side-effect of another feature like variable framerates/shutter speeds like ive talked about before.

Any significant DR extension using shutters would require multiple exposures at different shutters, and that would introduce motion artifacts. At least as far as I can tell.


@schauberger

Wouldnt your RGBU idea leave you with no color information at all for any of your extra shadow detail picked up by the U photosites (assuming no ND filtration on the RGB)? Even if you did filter the RGB, any extended range picked up by the U photosites wouldnt have any color information, right? wouldnt that look pretty odd?


@nick
What is this more dynamic range in the HPX-170 you speak of? Are you talking about dynamic range stretch? Isn't that just a gamma curve thats based on standard (linear rather than cine) gamma?


I still think an ND checker pattern over the bayer sensor is the most practical and simple solution since you dont have to worry about sync or extra stuff in your optical path. It would could integrate into existing cameras and require only one capture system and one capture medium.

 

[Nick Brian Walters]

@Noah I believe you're correct here, as I looked up some more info on this, but it does show that some manufacturers are working towards something. I don't think it's as simple as a curve for overall signal though, only for objects that are clipping.

 

[Noah Yuan-Vogel]

yeah thats what i meant, its basically an different knee function (maybe like autoknee?). not even totally sure why anyone would use a knee function when cinegamma looks so much better.

 

[Joe G.]

"NOW..There are allready 3d rigs commercially available that use the beam splitter setup, which could be used for aligning two cameras instead of offsetting them, you give one a bit more ND then the other, and you're ready to go....I'm oversimplifying, cause of course test shots would need to be done, processed, inspected, etc.....but this setup is possible with professional results."

Now that's something to try. Have any of the 3d Red guys tried this?

With prices dropping, like the new Scarlet fixed lens (although a prime would probably work best), this could be the future.

What are the limits of dynamic range with two cameras and the beam splitter route?

SOMEBODY must have tried it?

 

[David Mullen ASC]

Over-under 2-camera rigs like that have existed for decades for film cameras, so it's been possible for a long time to do that.

The fact that it hasn't been used for getting HDR images should tell you something, that other than for a few special circumstances, if a technique is not production-friendly, not easy to use, not affordable to use, and creates more data to record and needs more post time to process... it's not going to be adopted.

By the way "Benjamin Button" did a sort of simplified version of the 2-camera approach in the sense that for some shots, they did another take with the lens stopped down to record more detail in bright objects like table lamps and windows, so they had the option of taking those portions of the frame and pasting them into the original shot. But this turns these shots into efx shots (but since so much of the movie had efx elements in it anyway, it was not such a big deal to have these elements.)

 

[Nick Brian Walters]

@Joe

I havn't personally seen any beam splitter red rigs, so I'm unsure if anyone can try this with that kind of rig, but an offset system can still be lined up to produce good results(scarlet 3d rig pics come to mind)....I personally am unsure of what the stop gain is, but I've done real world tests with the HVX-200, and am VERY pleased with the results.....my tests were done in a generally low-light environment, and I was able to capture some light that would have otherwise been missed. I think that is where the most advantage can be seen.

I have personally seen great strides being done in tone mapping, noise reduction, HDR, and low-noise being done from hardware to software.....I think some day in the future, as long as we all keep pushing for it, I can see insane resolutions at GREAT light levels, with(and this seems an oxymorom), low overall lighting.

 

[Nick Brian Walters]

I also have to say that my goal isn't HDR that is insanely tone mapped...this seems to be the big fad with still photographers. My hope is to grab all the light from a scene without having to pull out the light meter every time I switch angles. Extreme tone mapping has it's place I think, but for the most part I'd definitely like to display an actor who walks from one interaction and goes to look out a window upon the world...without having to light the interior like it's inside hells inferno, or drop tons of stuff over the windows.....to have my shadows and my lit space, where it makes sense.

It seems natural to either shoot from outside and then show the landscape, or to light a small area and pan to the window, etc, etc, etc.....but with proper HDR we could just pan, PERIOD....the dark and the light, co-existing within a scene.

 

[plasmasmp]

I've done HDR tests with 2 JVC HD-250 cameras, with the same goal as you. Subtle tonemapping that makes the image pop and capture those extra details that are usually missing. I'd love to see this technology evolve even more than 3D because I think it would enhance the types of lighting situations we can shoot in as well as give more options on how the final image is presented to the audience

 

[Eki Halkka]

Nor do I feel that it is likely to be worth the hassle unless it is implemented completely transparently or is a positive side-effect of another feature like variable framerates/shutter speeds like ive talked about before.

The method i suggest would mean just turning on "EDR button" while shooting from the DOP point of view. Everything else would be done automaticly with firmware / post software.

In case you wonder, my cool new term "EDR" would stand for "Extended Dynamic Range", something between regular and HDR ;-)

 

[Noah Yuan-Vogel]

The method i suggest would mean just turning on "EDR button" while shooting from the DOP point of view. Everything else would be done automaticly with firmware / post software.

yeah thats what i was going to suggest you call it but didnt want to start that debate back up about what HDR means.

As it is RAW, it hardly seems feasible for such a thing to be totally transparent considering it would require recording at least 2x as much data for an increase of up to 1stop of DR. This would also require 2x as much processing to display a proper image in playback on set. Techs would find that their media offload times were automatically doubled, and editors would find that their playback/decoding is doubled as well.

Now that I think about it, might rolling be an issue as well since two open shutter frames will have the same exposure period and the end of the first frame will match the beginning of the second but there will be a full frame difference between the first line of the first frame and first line of the second or last line of the first frame and the last line of the second. This might mean that there would not only be the twice the skew (assuming constant pixel rate readout rate) but also exposure inconsistencies in areas of blurred motion.

 

[Eki Halkka]

As it is RAW, it hardly seems feasible for such a thing to be totally transparent considering it would require recording at least 2x as much data for an increase of up to 1stop of DR. This would also require 2x as much processing to display a proper image in playback on set. Techs would find that their media offload times were automatically doubled, and editors would find that their playback/decoding is doubled as well.

Yep, very likely so. That's why it should be an option that could be used if necessary, not something you leave always on by default.

Now that I think about it, might rolling be an issue as well since two open shutter frames will have the same exposure period and the end of the first frame will match the beginning of the second but there will be a full frame difference between the first line of the first frame and first line of the second or last line of the first frame and the last line of the second. This might mean that there would not only be the twice the skew (assuming constant pixel rate readout rate) but also exposure inconsistencies in areas of blurred motion.

If the shutter is true 360 degrees, it doesn't work that way, as far as i know.

As the "rolling shutter" (read-reset) takes actually a pretty long time, 360 degree shutter wouldn't be possible if the frame was always entirely read and reset before starting to expose the next frame.

Instead, when a pixel has been read for one frame, it will immediately start exposing the next frame - good for this EDR technique, but it may introduce interesting artifacts in general: because of this continuous exposure, some parts (i.e. bit of motion blur in the top) of "frame 2" may actually be from an earlier point of time than some parts (i.e. bit of motion blur in the bottom) of "frame 1".

 

[Eki Halkka]

By the way "Benjamin Button" did a sort of simplified version of the 2-camera approach in the sense that for some shots, they did another take with the lens stopped down to record more detail in bright objects like table lamps and windows, so they had the option of taking those portions of the frame and pasting them into the original shot.

I've used that technique pretty often, for simple shots that don't require too much work in post (i.e. fixed shots). Works great.

 

[ColinSmith]

Once you start shooting 360 shutter and working the frame stream in post you have a bunch of options and dehree of freedom for choosing how to use the images in post.

Essentially shooting Xfps at 360 shutter gives you post "exposure options" running "downwards" from there in both directions - I've shot video sections to drop into timelapse comps posted to give a 25fps stream with each post-post frame having a 1s exposure, just by shooting 25fps, 360 shutter and doing a rolling blend for 25 frames.

Lots of possibilites, just by thinking a little bit outside the usual film camera setups to see some of them.

 

[Joe G.]

"Once you start shooting 360 shutter and working the frame stream in post you have a bunch of options ..."

Do you have some test reels that show these various possibilities on the Red?