Shooting in log?

[Stu Maschwitz]

Well, that's sort of my question -- is it even possible to read / measure that (clip point) in a RAW file, live, without converting it to RGB?

Maybe not, but imagine this: Imagine there was only one RGB conversion in the camera, and it was, let's just say, full-range chip mapped to RED Log. Always, no matter what, that's what the camera sends out to display. With, of course, the option to apply a 3D LUT.

You'd have default 3D LUTs for Rec709 and other common video standards. The user could create 3D LUTs for any conceivable use, including filmout to 35mm. And for any given LUT you might like to shoot with, you could also have a version with exposure assist overlays (this can actually be baked right into the LUT). Thos overlays could include a true slipping warning.

The one-size-fits all RED Log conversion would need to be of a high enough bit-depth within the camera to withstand some manipulation, but the subsequent output could be good old video.

This, it seems to me is where the camera should head. If people are to use a camera for digital cinema, 3D LUTs will be a necessity. And those LUTs will be meaningless without a common colorimetric basis.

-Stu

 

[David Mullen ASC]

Maybe not, but imagine this: Imagine there was only one RGB conversion in the camera, and it was, let's just say, full-range chip mapped to RED Log.

What would be the particulars? Are you saying a live conversion to 10-bit LOG 4K RGB, and then another conversion for HDSDI-out? Or 10-bit LOG 1080P or 720P 4:4:4? Or 4:2:2?

Aren't we getting into some serious live conversion and processing requirements?

 

[Stu Maschwitz]

I was thinking one output of 10-bit 4:4:4 1080. 3D LUT processing could be outboard with a signal like that—works for the Genesis. What would you want it to be?

-Stu

 

[Stu Maschwitz]

Not if you feed it to an external monitor that can either implement a view LUT (like a Cinetal) or a box that does the same thing (like a LUTher). I can't speak for Stu, but that could be what he's thinking here: a way of getting a reasonable film print preview during production. This is a common way of working with cameras such as the Viper and Genesis.

And now we're right where mmost was on page one.

-Stu

 

[Michael Bach]

Technical Monitoring

Technical Monitoring

All disscusion points to the same requirement.

We are shooting with a RAW camera. The object of the exercise is to capture all available data in RAW mode to enable the DP/post prod people to choose the bit they want in Red Alert / Cine.

So pretty much the only thing we need is an accurate method of deciding whether the sensor is beyond its limits particularly at the white end of exposure. The DP needs to know if he has to adjust his lighting etc to ensure that the whites are not being clipped in any way.

So we either need a "once and for all" ISO rating for the sensor and chain which enbles some degree of confidence with the light meter and/or an electronic tool in the viewfinder which says to the user "hey you are losing detail in the whites"

Using the monitoring outputs is obviously pointless as they are "tweakable" from within the camera setup.

Rather than massively increasing the amount of internal processing that the camera has to do by putting in 3D luts and selectable "standards", how bout something that measures high frequencies in the whites which displays as a graph in the viewfinder? Ih there is no high frequency information in the whites, you are clipping and hence losing data. Sorry guys, not trying to re-invent the wheel on a pretty clever camera but we need to have some way of assuring that the sensor is in its happy place

Cheers
Michael

 

[Dj Joofa]

Rather than massively increasing the amount of internal processing that the camera has to do by putting in 3D luts and selectable "standards", how bout something that measures high frequencies in the whites which displays as a graph in the viewfinder? Ih there is no high frequency information in the whites, you are clipping and hence losing data. Sorry guys, not trying to re-invent the wheel on a pretty clever camera but we need to have some way of assuring that the sensor is in its happy place

Cheers
Michael

LUTs are one of the easiest things to put in software/hardware mapping. Mappings via functional forms, or doing Fourier type analysis such as one you are describing (high-frequency detection) are more difficult.

Bottom line, LUTs are damn easy that is why they are there.

 

[Michael Bach]

To put it in very simplistic terms (because there are people here far smarter than me who can explain the technical ins and outs) at the end of the day any Sensor must have a minimum and maximum design characteristic....that is it will be sensitive a certain minimum amount of light and it will be overwelmed or fully excited by a certain maximum amount of light. By defining mins and maxs we get a "real" dynamic range.

The "once and for all" ISO value for the sensor and chain should not be adjustable nor theoretical...it should be tested and proven...in fact I am sure that Graeme and co know exactly what it is and equally I am sure that they have good reasons for not publishing to date. (mostly cause its not finished yet).

Red has brilliantly designed a system where potentially more dynamic range can be captured than is needed for the rec709 world but (currently) less than film can deliver. I am sure that sooner or later the performance of the system will allow more DR to be achieved.

However in the meantime, in order to correctly expose the camera, a suitable method, which does not involve the monitoring outputs, must be found to ensure that the sensor is not outside its design parameters (unless you want it to be) when the red button is pressed.

Cheers

Michael

 

[Arnaud Paris]

Stu;
The 1.5 stop left in the brights I was referring to is the difference between the REC709 clipping and the raw clipping.
And this is leaving the camera setup at 320ASA no matter what. It's just that the DP lights his for in between 160 and 250; and he knows that whatever is clipping on the Red LCD still has 1.5 stops worth of information left in the raw file.
Of course if you change the camera preview mode to 160ASA then it's a different story...

 

[Miguel "Macgregor" De Olaso]

My 2 cents. Over the last 4 weeks i´ve been playing with these ideas you are now putting over the table.

With the actual build 14, at 320 ISO on camera, and exposing with false color the 709 signal:


- RED has +4 stops over 18%grey
- RED has about -6 stops under 18%grey, but the last 3 have a lot of grain.

- Stu´s approach of exposing as the Genesis/f23 do, setting the 18% grey 3 stops underexposed to have +5 stops over 18% grey sounds nice.
It doesn´t work in real world. Too much noise. Pattern noise.

- At a given scene, there is a difference of at least 1.3 stops between the false color and the cameraISO.

- For those who want to expose using a light meter set at 320ISO using a RED ONE set at 320 ISO (in camera 320ISO seems to be the best balance between S/N ratio) you will get an image underxposed in the LCD as discussed before (even when this "might" be good to preserve highlights and make a knee adjustment in post in order to have a "natural" transition between highlights and overexposed areas).
But how can the camera man operate such a dark image. You can increase the ISO up to 640-800ISO, keeping the lightmeter set at 320 and manually changing in redcine each clip down to 320ISO (or even 250 to prevent REDCINE 0.4 clipping). A pain in the ass.

- You can all argue asking to have more color space and gamma outputs. Sure this could help a bit, but the sensor will still record the same information.

- Either we accept a camera that only has +4 stops over 18% grey, or you deal with huge amounts of noise. So much noise that no one will ever accept it.

 

[Stu Maschwitz]

Stu;
The 1.5 stop left in the brights I was referring to is the difference between the REC709 clipping and the raw clipping.
And this is leaving the camera setup at 320ASA no matter what. It's just that the DP lights his for in between 160 and 250; and he knows that whatever is clipping on the Red LCD still has 1.5 stops worth of information left in the raw file.
Of course if you change the camera preview mode to 160ASA then it's a different story...

Yeah, I understand Arno but I don't see it in my own testing. We shot at 320 and when I set RED Alert to Rec709 and default settings, I see exactly what we saw on set. Then I can pull a little less than 0.4 stops of additional information by reducing exposure.

-Stu

 

[Stu Maschwitz]

With the actual build 14, at 320 ISO on camera, and exposing with false color the 709 signal:

- RED has +4 stops over 18%grey

Really? You're saying that in addition to the +2 stops over 18% gray that you can see in Rec709, you're finding an additional two stops that you can pull in using RED Alert?

That is so very different than what I'm seeing with our footage.

-Stu

 

[David Mullen ASC]

That doesn't sound too odd -- a Caucasian face is one-stop over 18% grey (I seem to recall 18% grey is Zone 5 and one-stop over would be Zone 6). My brief, unscientific test of RED showed clipping occurring about two-stops over the Caucasian face in the frame, i.e. three-stops over 18% grey, and that was looking at the 720P REC 709 image on the set. I assume nearly another stop of info is available in the RAW file, maybe not.

This was at 320 ASA.

Most HD cameras crap out and get clippy-looking if a Caucasian face is two stops overexposed.

 

[Miguel "Macgregor" De Olaso]

Definately.

Check this chart:

Latitud means dynamic range in spanish.
Zona de ruido digital, means digital noise area.

XD

In the end, we should be all talking about zone system, and not so much about ISO. Where do we want to set our 18% grey is the clue.

 

[Stu Maschwitz]

Are each of those patches really a full stop? I'm not really sure what I'm looking at.

If you look at RED's documentation of the false-color mode you see something different, note that they consider 18% gray to be 3 stops below clip, which is actually a little more than the formula for Rec709 would strictly indicate.

-Stu

 

[Stu Maschwitz]

...Where do we want to set our 18% grey is the clue.

Indeed, where did you put it in your testing?

I ask because the Rec709 maths indicate this:

If you put 18% gray at 45% Rec709, then 1 stop over 18% maps to 65.1% image value, 2 stops over maps to 92.6%, and 3 stops over maps to 130.1%, which would be clipped.

I'm trying to resolve the apparent differences between the strict Rec709 formula and the results people are reporting.

-Stu

 

[McDiver]

Stu, Love your blog. And this discussion has to be ranked up there with the most valuable on RedUser ever.
But I'm confused about the Rec709 being a problem.

For Red, isn't it just a non-destructive tool with which to evaluate how much light is falling on a chip? Isn't Rec709 just a dirty little window to see what's happening inside the Red camera. It can't claim to reveal all, but lets you see enough?

And, as a video standard, is it practical to expect it to see all raw data? You can under-expose in AE, but this camera does get noisy at the dark end.

Also, if mapping gray to 45% means 3 stops is mapped to 130%, how is that a problem? Rec709 isn't dictating what gets recorded. It's just a window. We're already in agreement that there's almost half a stop available above the "apparent" clipping point seen in Rec709, which would certainly accomodate the 130%, wouldn't it?

Granted, this doesn't help monitoring absolute highlight clip points, and I think they're working on a better solution. But I like MacGregor's suggestion. Bump up the ISO for monitoring, and then ignore it for actual metering. Kinda sucks you have to reconvert ISO later again. Perhaps an "apply this setting to all clips" feature could be added?

It'd be nice if the LCD could get an ISO setting independent of the metadata, as a quick fix before custom LUTs and False Color become available.

If I'm not understanding you, I apologize in advance.

 

[reality]

Canon DSLRs work in a similar fashion where monitoring and histogram are reflective of the processed image rather than the raw, unprocessed input.

Both raw and processed workflows are equally valid IMHO. The discussion does seem academic considering the suggested changes in builds 15 and 16.

With respect to cineon mapping, Shake and other programs default their log2lin and lin2log conversions to what's in the old cineon docs. Those conversions curves do significantly limit the curves to not much above 685. While most film scans have density that go up into the 900s, some apps like shake by default don't do constrict lin2log conversions to ~685. Shake does the same for log2lin in integer mode but not float which works properly.

I've found that PDlog685 in RedAlert linearizes nicely but Redlog is nicer because it uses the full range. The colorists I've encountered prefer the PDlog685 setting except for the more adept which prefer the full range redlog or pdlog9xx. I feel redlog works perfectly for me. It gives me the full range when I prep output so the histogram in redalert is not clipped or raised too much.

 

[Dj Joofa]

- For those who want to expose using a light meter set at 320ISO using a RED ONE set at 320 ISO (in camera 320ISO seems to be the best balance between S/N ratio) you will get an image underxposed in the LCD as discussed before

And,

I ask because the Rec709 maths indicate this:

If you put 18% gray at 45% Rec709, then 1 stop over 18% maps to 65.1% image value, 2 stops over maps to 92.6%, and 3 stops over maps to 130.1%, which would be clipped.

Guys, I think Rec 709 is being abused for what it is not intended for. Just as LAB space is abused for what it is not intended for.

Reading what Macgregor is reporting that metering using a light meter at 320 ISO and then setting up a Red one at 320 ISO, results in a mismatched (underexposed LCD) behavior, to me it appears this mismatch may resulting from how Red tries to obtain a perfect correspondence between a film world notion of ISO and a digital world ISO equivalent.

There are *several* formulae available for computing digital ISO equivalent and many manufacturers are ambiguous on the following quantities used to measure the digital ISO equivalence:

(1) Which formula for ISO was used?
(2) Was any up/down conversion used to round the calculated digital ISO to standard film world ISO?
(3) Was the digital sensor ISO calculated with or without the lens effects?
(4) Was analog gain *after* sensor and *before* ADC kept constant; even if kept constant, was it at its lowest value?
(5) How was noise measured? -- RMS, peak-peak, etc.
(6) How was signal measured in relation to noise?
(7) And the list goes on and on.

I haven't read any Red manuals and hence I do not know if any of these questions are answered in there. However, to me it appears it is difficult to come up with exact 1-1 correspondence with film world notion of ISO and digital sensor notions of ISO, and hence, if metering was used in the standard film world way, then does it necessarily translate to the exact same metering values for the "same" digital ISO?

If not, then we may be blaming the Rec 709 space for something it may not be liable to.

 

[GlennChan]

I don't think it's necessary to get overly technical here.

Why not do a test where you bracket your exposures and simulate the end product? Shoot, grade, figure out what exposures work for you. If you are concerned about the end result (and not S/N ratios ISO ASA ADC RMS whatever) then this test is probably the closest to giving you the answers you're looking for.

 

[Dj Joofa]

I don't think it's necessary to get overly technical here.

Why not do a test where you bracket your exposures and simulate the end product? Shoot, grade, figure out what exposures work for you. If you are concerned about the end result (and not S/N ratios ISO ASA ADC RMS whatever) then this test is probably the closest to giving you the answers you're looking for.

GlennChan,

Non sequiturs happen frequently in the technical world. If we do a bracket/exposure test, it is entirely possible that you and I reach the same conclusion. However, when we look back and try to offer theories (Is Rec 709 is to blame?; Is this a film vs. digital metering problem?, etc.) as to why that certain conclusion was reached, then it is possible that some of our different reasonings were flawed, even if we reached the same conclusion.

The objective of any knowledge impartation whether acquired through formal education or by personal experience is not to apply that same things which one has learned, but to impart analytical reasoning skills that shall help us in identifying if the relationship of the reasoning vis-a-vis conclusion is sound or not.