LUT Education

[Mark L. Pederson]

one important one being that status quo seems to win over innovation far to often.

Battistella

I believe that if the innovation in our industry provides enough benefit - savings of time & money being the key driver - but not the only driver of adoption of course - it will win in the end.

It takes education - and it takes a loud voice (aka "evangelist" aka "marketeer") and it takes a few risk takers (aka "early adopters").

Typically, if the early adopters achieve success with the innovation - they become the evangelists. The "thought leaders".

In our industry - CASE STUDIES are key. Everyone likes to follow.

"Fincher edits a feature on Premiere - it worked for him - we should consider editing on premiere", etc.

It just takes time. You can't stop it - it's coming - it's only a question of how soon.

Want a 9K+ full frame sensor? It will happen. The only question is how soon.

I remember when a 4k camera was scam. I was in that booth at NAB in 2006 and people really did laugh out loud at the whole RED deal.

The status quo is VERY over rated. The future is REALLY exciting.

 

[Hrvoje Simic]

I have heard some major-name DPs complain that the problem with trying to set up complicated LUTs and looks on set is that it drags a lot of time away from their actual photographic process, and it slows down the whole shoot. .

Understandable.


That is about to change.

 

[David Battistella]

Great to see you here hrvoje!

battistella

 

[Evangelos Achillopoulos]

Here's a picture of the three (unrelated) Shaws at IBC – courtesy of Warren Eagles:

Left to right, Nick Shaw, Steve Shaw and Kevin Shaw

Finally faces to the names...

 

[M Most]

The status quo is VERY over rated.

I don't disagree. But experience and knowledge are NOT overrated, and there is, in many caaes, far too little of both in the DIT arena, at least regarding color. Creating complex 3D LUTs, and knowing how and when to use them, is not for those who somewhat randomly change multiole settings "until it looks right," without any real understanding of what controls to use, in what order, and how far one should go before they're doing more harm than good. I've seen situations where the color settings were severe enough that the exposure ceased to have meaning. And the less disciplined the DIT, the more likely that is, Original photography is NOT about wacky electronic manipulation. It is about proper exposure, proper lighting, and control. Previewing final looks is fine, but only if one really understands what they're actually capturing.

Regarding CDLs, it is a bit of a misnomer to think that their primary use is to pass on settings that will be used in final grading. This is, as Mark correctly states, rarely the case. The primary use of CDLS - at least in big features - is to achieve a degree of visual consistency in visual effects. This is important because visual consistency in the offline edit makes for better screenings and less distraction for both the editorial team and the director when looking at cuts. As long as a pipeline is set up that ensures that debayered files are supplied to VFX vendors that, when the CDL and whatever back end LUT is being applied look essentially the same as dailies produced for the rest of the scene, that consistency is achieved. Once the show is in the hands of the final colorist, dailies have meaning only as a reference to a look that is sometimes maintained in the final piece and sometimes not. But the real value of the CDLS generally comes in the middle of the post process, not at the end.

 

[Nick Shaw]

…there are workflows that use ONE 3D LUT for a creative look - and then CDLs ride on top for "best light" adjustments for dailies

In fact it is often useful to put CDL adjustments before a LUT. Because a LUT designed using a particular image as a reference will work best if the image coming into it has a similar distribution of pixel values to the reference image.

To a great extent, if you are monitoring through the LUT you are getting the image into the right ball park for the LUT's input by adjusting your lighting and expoure, but a CDL type adjustment on the input of the LUT can improve the situation.

A 3D LUT can paint the entire canvas of a camera or footage's color matrix, providing multiple digital signal interpretations and it can provide more elaborate, direct modification abilities of chrominance and luminance without the need of qualifiers - without dismantling the image - and all done in realtime with less destructive manipulation.

The main limitations of a 3D LUT are:
1. A 3D LUT is a "black box" where you put image data in, and get different image data out. If you only have the LUT itself, you have no idea how the transform is constructed, and no ability to tweak it for a particular image.

2. It cannot do anything spatial, so any grades with a spatial component cannot be represented by a LUT. It may not be appropriate to try and create one-size-fits-all power windows, but vignettes and blurred qualifiers are two frequently used tools in grading, and neither can be represented by a LUT.

3. Different implementations may give slightly differing results with the same LUT, depending on the interpolation method used. Higher mesh densities (such as the 64^3 cubes Mark is proposing) make this less of an issue.

Resolve Live seemed like a potential solution to these, but the inherent latency makes it unsuitable for putting in the monitoring pathway on a shoot.

FilmLight have a very good solution to this in Flip, but unfortunately not every production can justify the cost of one. The new version of Flip I saw demoed at IBC was smaller and cheaper (I don't think actual pricing was announced) and has a number of additional useful functions.

For those who don't know about it, you can read about Flip here. But that page shows the old version, not the new much smaller one shown at IBC. The new Flip is controlled by iPad, which can also show live images, and can apply a Baselight grading stack (incuding spatial operations) to a live image, with negligible latency.

This means you are applying the exact operations to the live image that you use as the start point for the final grade, but rather than being a "flattened" LUT you have access to all the elements that make up the look, and can tweak each part on a per shot or per scene basis if required. This is all saved as metadata, and can be applied automatically to the right shots in the DI.

I hope to be able to test one properly soon, and will write up my findings.

The budget solution is to have your colourist create your looks in whatever grading system you will use for the DI, and save identically named 3D LUTs and saved grades for each look. The 3D LUT will be an approximation of the grade, with any spatial components removed. You monitor using the LUT on the shoot, then appply the look to dailies, either with the LUT, or using the saved grade if you can use the same grading system for dailies as will be used in the DI. You then need to track the looks through editorial, so you can automatically apply the grades in the DI conform, using the full stack/node tree, not the LUT.

I have a demo of a workflow similar to this (not using a RED camera) here.

 

[Nick Shaw]

I've seen situations where the color settings were severe enough that the exposure ceased to have meaning.

Mike makes a very important point here. Monitoring via any processing (and this applies equally to LUTs and raw settings) can affect exposure decisions. If people don't understand what they are doing they can end up recording a badly exposed raw file, just to make it "look good" through the LUT.

 

[M Most]

This is all saved as metadata, and can be applied automatically to the right shots in the DI.

Well, yes, but with the very significant limitation that your DI must be done on Baselight. I'm as much a fan of Baselight as anybody, but what's needed is generalized solutions to color management, not system specific ones that require proprietary solutions in order to work. That is exactly what The CDL was created to solve. Relying on pipelines built around a specific vendors solutions is exactly what encourages the kind of disinformation passing that Mark referred to earlier. When information is based on a sale, it is colored by what the specific solution has to offer. When information is based on coming up with the best pipeline, it can draw on all of the best solutions for the particular tasks involved (dailies, VFX, DI, etc.). But that only works with generalized glue pieces that can tie those solutions together, and are supported by multiple vendors.

I would also comment on the "3d LUT vs. CDL" notion that seems to be pervading this thread. CDLs and 3D LUTs are not mutually exclusive. They are usually used together in a color pipeline. The 3D LUT is used to achieve a proper color space conversion, a "look" application, or both, and is commonly placed at the back of the color pipeline. The CDL is used to modify the camera output (whether it be live or from the recorded file) for both exposure and color balance in order to "normalize" it to the values that the 3D LUT expects (since the LUT cannot be modified). The fact that some people seem to think the LUT should be prior to the CDL in the pipeline only serves to illustrate exactly what I said earlier about the level of experience and knowledge being far lower than it should be.

 

[Nick Shaw]

Well, yes, but with the very significant limitation that your DI must be done on Baselight.

True. But that is to some extent inherent in a sophisticated system. If you take a parametric approach, you are limited to using a system which understands those parameters. Although it is possible to convert the operations in a BLG into e.g. a Resolve DRX if you work out the underlying maths.

ASC CDL is a known standard (given that you have defined for your pipeline what colour space it is applied to and a what point in the processing order, and in combination with what LUT) but is limiting (some would argue in a good way). You cannot construct an s-curve with the SOP function, so if you are to avoid clipping image data at both ends, you need to use it in conjunction with a LUT which applies a tone-mapping curve.

The fact that some people seem to think the LUT should be prior to the CDL in the pipeline only serves to illustrate exactly what I said earlier about the level of experience and knowledge being far lower than it should be.

Absolutely. And it worries me that the lack of knowledge and experience you refer to is not limited to the lower end of the market.

 

[Mark L. Pederson]

I would also comment on the "3d LUT vs. CDL" notion that seems to be pervading this thread. CDLs and 3D LUTs are not mutually exclusive. They are usually used together in a color pipeline. The 3D LUT is used to achieve a proper color space conversion, a "look" application, or both, and is commonly placed at the back of the color pipeline. The CDL is used to modify the camera output (whether it be live or from the recorded file) for both exposure and color balance in order to "normalize" it to the values that the 3D LUT expects (since the LUT cannot be modified).

Yes. Agreed.

But .... in MOST of the current "status quo" workflows - the 3D LUT is ONLY used for color space conversion and/or monitor matching - certainly true in episodic TV.

What I want to do is open more people's eyes to what you can CREATIVELY with a 3D LUT - such as the workflow of Creative 3D LUT > CDL > Display LUT.

And a Creative 3D LUT can - and will be modified - in future workflows.

 

[Gavin Greenwalt]

The fact that some people seem to think the LUT should be prior to the CDL in the pipeline only serves to illustrate exactly what I said earlier about the level of experience and knowledge being far lower than it should be.

It somewhat depends on your LUTs intent and use. I have a LUT I've built out of FilmConvert which goes from REDLogFilm -> "Kodak Vision" -> REDLogFilm w/ "Kodak" Color. From there you can apply all of the regular CDL operations as if it were a flat image. In that regard I'm treating my 3DLUT more like REDColor in fact I'm just converting from DragonColor to KodakColor not as a "look" with full response curve exposure modification it's almost exclusively a chroma operation.

 

[Mark L. Pederson]

1. A 3D LUT is a "black box" where you put image data in, and get different image data out. If you only have the LUT itself, you have no idea how the transform is constructed, and no ability to tweak it for a particular image.

But what if you had an easy to use tool that did a decent job of graphically representing how the transform was constructed - and an intuitive toolset allowing you to "sculpt" the 3D LUT and tweek it

 

[Mark L. Pederson]

It somewhat depends on your LUTs intent and use.

Not somewhat - COMPLETELY.

Again, the "status quo" workflows are going to change when more people embrace 3D LUTs for CREATIVE - not just a color space conversion and/or display LUT.

 

[Hrvoje Simic]

The main limitations of a 3D LUT are:
1. A 3D LUT is a "black box" where you put image data in, and get different image data out. If you only have the LUT itself, you have no idea how the transform is constructed, and no ability to tweak it for a particular image.

Agreed.

Which is why it is beneficial having another type of "box" with it.

2. It cannot do anything spatial, so any grades with a spatial component cannot be represented by a LUT. It may not be appropriate to try and create one-size-fits-all power windows, but vignettes and blurred qualifiers are two frequently used tools in grading, and neither can be represented by a LUT.

Those factors have to be adjusted per shot basis, while hue, sat & gamma do not.
At least nowhere near that extent.

Creative 3DLUT does not have to carry any spacial or blur or any other image de/re-construction related data.
That is just topping.

What 3D LUT can carry is the digital essence of the image aesthetic.
Precious for a cinematographer. Getting the feel of his creation. While he is painting it.

3. Different implementations may give slightly differing results with the same LUT, depending on the interpolation method used. Higher mesh densities (such as the 64^3 cubes Mark is proposing) make this less of an issue.

That is a factor with pipelines which use differing implementations and workflows. Sticking with those explored, tested and relied upon removes these factors out of the equation.

 

[Gavin Greenwalt]

Ultimately the best "LUT" wouldn't be a LUT but something like OFX, HLSL or BINK. A way of describing any image manipulation but also exposing parameters. You can then build a "LUT" but also include 2D spacial effects like blur.

The reason I say this is because while LUTs are important another important dimension to the image that will soon be going mainstream is digital filtration. Once we get a camera that delivers 20 true stops of usable dynamic range cinematographers are going to hopefully move to post-diffusion just like they've moved to post white-balance from CTO and CTB in the mattebox.

 

[Mark L. Pederson]

Once we get a camera that delivers 20 true stops of usable dynamic range cinematographers are going to hopefully move to post-diffusion just like they've moved to post white-balance from CTO and CTB in the mattebox.

BINGO. 100% will happen. Smartest guy in the room as usual.

 

[Nick Shaw]

Creative 3DLUT does not have to carry any spacial or blur or any other image de/re-construction related data. That is just topping.

I am not talking about bluring the image itself, but rather bluring the matte from an HSL qualifier, as is often done to clean up the key.

A 3D LUT will always map the same input triple of RGB values, to the the same output triple. Grades with secondaries, with a blur applied to the qualifier matte do not have the same one to one mapping. So a LUT implementation of a HSL qualified secondary cannot have the matte blur applied, so may well give a noisier result.

Ultimately the best "LUT" wouldn't be a LUT but something like OFX, HLSL or BINK. A way of describing any image manipulation but also exposing parameters. You can then build a "LUT" but also include 2D spacial effects like blur.

Somebody approached me a couple of years ago and asked me what my ideal LUT box would be like. My reply was that what I would want would basically be a GPU in a box, so you could load fragment shaders to implement transforms. Sadly nothing ever came of it.

I'd love to have such a box. And don't forget, fragment shaders can include texture lookups, so you can still have a LUT as part of the pipeline for operations not expressible mathematically, such as measured film stock responses or display profiles.

That is actually essentially what a Flip is, except instead of directly loading fragment shaders you have the Baselight UI.

 

[Hrvoje Simic]

I am not talking about bluring the image itself, but rather bluring the matte from an HSL qualifier, as is often done to clean up the key.

A 3D LUT will always map the same input triple of RGB values, to the the same output triple. Grades with secondaries, with a blur applied to the qualifier matte do not have the same one to one mapping. So a LUT implementation of a HSL qualified secondary cannot have the matte blur applied, so may well give a noisier result.

Yup.

Looks like the ability to modify specific H/S/L values without a typical HSL qualifier...would be beneficial.

 

[M Most]

But what if you had an easy to use tool that did a decent job of graphically representing how the transform was constructed - and an intuitive toolset allowing you to "sculpt" the 3D LUT and tweek it

Although that might sound good, considering the current level of understanding and competence, I have grave doubts as to who exactly is going to be the person doing such manipulations. Because you're no longer talking about cinematography, you're talking about digital image manipulation that really, really, honestly, doesn't belong on a set where you're far more concerned with making the day, dealing with things that go wrong, dealing with getting the best performance, and wrapping the location. I honestly don't understand the desire to do time consuming, meticulous work in an environment where, by definition, you have the least time available than at any other point in the process.

 

[Gavin Greenwalt]

Although that might sound good, considering the current level of understanding and competence, I have grave doubts as to who exactly is going to be the person doing such manipulations. Because you're no longer talking about cinematography, you're talking about digital image manipulation that really, really, honestly, doesn't belong on a set where you're far more concerned with making the day, dealing with things that go wrong, dealing with getting the best performance, and wrapping the location. I honestly don't understand the desire to do time consuming, meticulous work in an environment where, by definition, you have the least time available than at any other point in the process.

This falls into the category of "If it's hard you're doing it wrong." Software should be easy. If it's slowing things down then it's been done poorly. For instance a lot of people pick DragonColor and that's a 3DLUT. Other people prefer older colors like REDColor2, that's a LUT. It doesn't slow down the workflow to prefer one over the other but the output is pretty different. Similarly picking Kodak over Fuji, if you could monitor directly without running dailies would have an impact on how the cinematographer reviews a setup. I agree you shouldn't be pixel fucking on set but a quick tweak on the shadow and highlight rolloff could be done in 2 seconds for a setup.