Shaper LUTs vs. CDLs

[Michael Tiemann]

Across a variety of threads I have tried to promote the idea that CDLs should be better supported as first-class objects within the proper support of IPP2, rather than special-cases relegated to ColorTrace. I've had little traction on Blackmagic's Resolve forum (Resolve is perfect as it is! RED IPP2 should not be a first-class citizen!). The other day I came across the abomination known as "Shaper LUTs" (pages 870 and 889). This is an idea of using 1D LUTs (which can have lots of 1D precision) to pre-condition data so that 3D LUTs don't clip too badly in their first or last buckets.

To that I say: this is why CDLs exist! The CDL provides a mathematical transfer function--which is infinitely more precise--than a 1D shaper LUT.

So I again say this to the intersection of REDusers who happen to use Resolve: let's get a first-class, first-rate implementation of IPP2 support in Resolve, with all the right stages ordered in all the right places, and we'll really have something to write home about!

 

[Jake Wilganowski]

if resolve never fully supports IPP2 in its proper implementation that is a big problem

 

[rand thompson]

Michael,


Here was a "work around" I proposed for the IPP2 workflow in Resolve. It's not the implementation you and I and everyone else actually wants from resolve, but just the best I could come up with.


http://www.reduser.net/forum/showthread.php?165235-DaVinci-Resolve-14-3-is-now-released/page4

 

[Aaron Lochert]

If Blackmagic doesn't want to go through the proper channels to give us an IPP2 color managed project workflow similar to how ACES works, then the least they could do is take the same output transformations/tone mapping math from the SDK and either create a specific OFX plugin for that or just update the Color Space Transform plugin.

That would allow us to decide if we want to grade on top of Rec709-ish stuff right out of the gate, or implement it properly ourselves with the output at the, well... output. And this avoids having to use any output LUTs.

 

[rand thompson]

If Blackmagic doesn't want to go through the proper channels to give us an IPP2 color managed project workflow similar to how ACES works, then the least they could do is take the same output transformations/tone mapping math from the SDK and either create a specific OFX plugin for that or just update the Color Space Transform plugin.

That would allow us to decide if we want to grade on top of Rec709-ish stuff right out of the gate, or implement it properly ourselves with the output at the, well... output. And this avoids having to use any output LUTs.

Agreed!!

 

[Phil Holland]

Proper IPP2 implementation is coming in Resolve and other places. It's just taking a while due to too many factors to list.

I will say some of this feels a bit like the good old HVX200 days where in the first year and a half you had to use the Raylight codec to even work with the footage on a PC.

But yeah, it's coming soon to everything you'd expect shortly. Hopefully all done rather well.

 

[Aaron Lochert]

When that happens, I just hope Michael goes back onto the Resolve forums and gives a little jab to the RED haters.

 

[Marc Wielage]

When that happens, I just hope Michael goes back onto the Resolve forums and gives a little jab to the RED haters.

I hang out in 3 different Resolve forums and have seen very little animosity towards Red cameras in the last 3-4 years. My own feeling is that about the only cameras that get my hackles up are cheap 8-bit H.264 cameras (particularly DSLRs), but Red, Alexa, the 10-bit Canons, the Panasonic Varicams, and the higher-end Sonys are all capable of great pictures. Those are generally very well-liked, provided they're in the right hands. I'm on a Sony F55 feature right that took a little time to get it right, but I'm happy with how it looks. The last major Red project I did was fantastic and the DP did a superb job (just premiered at a festival over the weekend).

 

[Aaron Lochert]

I hang out in 3 different Resolve forums and have seen very little animosity towards Red cameras in the last 3-4 years. My own feeling is that about the only cameras that get my hackles up are cheap 8-bit H.264 cameras (particularly DSLRs), but Red, Alexa, the 10-bit Canons, the Panasonic Varicams, and the higher-end Sonys are all capable of great pictures. Those are generally very well-liked, provided they're in the right hands. I'm on a Sony F55 feature right that took a little time to get it right, but I'm happy with how it looks. The last major Red project I did was fantastic and the DP did a superb job (just premiered at a festival over the weekend).

Oh, yeah. In general RED is getting more and more respect, which is great to see. I was more-so alluding to a specific instance when Michael had asked for proper IPP2 support on the Blackmagic official forums (in the Resolve feature request thread, of all places) and was shot down and labeled as a "fanboy."

 

[Steven Abrams]

If Blackmagic doesn't want to go through the proper channels to give us an IPP2 color managed project workflow similar to how ACES works, then the least they could do is take the same output transformations/tone mapping math from the SDK and either create a specific OFX plugin for that or just update the Color Space Transform plugin.

This is not how SDKs in general work. The app developer does not get access to the code or math (unless the SDK maker decides to provide it) - they have access to interfaces (functions) that they can call from the library (they don't know the code for the functions either, they can just pass and receive data via them). It would be quite inefficient for the red sdk to decode and pass a processed image to resolve, resolve do processing (aka you do grading), and then pass that image back to the red sdk for more processing just for some tone mapping, then pass that back to resolve for output etc. Really not good for memory and passing data around. If red decided to give blackmagic their tonemapping math then they could more practically add it (and theoretically for any camera footage since it would be odd to add just for one camera manufacturer if you have mixed camera sources for example), but if they haven't then it's on red.

 

[Michael Tiemann]

This is not how SDKs in general work. The app developer does not get access to the code or math (unless the SDK maker decides to provide it) - they have access to interfaces (functions) that they can call from the library (they don't know the code for the functions either, they can just pass and receive data via them). It would be quite inefficient for the red sdk to decode and pass a processed image to resolve, resolve do processing (aka you do grading), and then pass that image back to the red sdk for more processing just for some tone mapping, then pass that back to resolve for output etc. Really not good for memory and passing data around. If red decided to give blackmagic their tonemapping math then they could more practically add it (and theoretically for any camera footage since it would be odd to add just for one camera manufacturer if you have mixed camera sources for example), but if they haven't then it's on red.

At the risk of saying something completely uninformed, but based on years and years in the software industry and an author of major tools for writing SDKs, I will say this about that. RED has published that IPP2 has three stages of image processing. They have indicated that there are multiple things that happen at each stage. I would therefore expect that each stage has its own first-class interface that effectively forces a round-trip between the user and the SDK, such as:

// start stage 1 processing
raw_image->stage1();

// carry on with stage 2 processing
raw_image->stage2();

// finish up with stage 3 processing
raw_image->stage3();

There is absolutely no reason that Resolve, Premiere Pro, or anybody else, couldn't or shouldn't do whatever else they want to do before Stage 1, between Stage 1 and Stage 2, between Stage 2 and Stage 3, or after Stage 3.

By the same token, I would agree that internal elements of each stage, such as Kelvin processing as one item or Tint processing as anyother--those may effectively be atomi in nature, providing no interior access nor encouraging any interstitial processing.

I'm pretty sure that the real problem of the NLEs is this: they all have a function that's effectively Process_Raw_Image as a single, monolithic thing. They are ganging together all three stages in that single function call, which was fine and dandy when RAW processing was just debayer->Kelvin->Tint->Color Space->Gamma. But IPP2 makes that more complicated/modular. And indeed it would be basically impossible for ANYBODY to get it right if RED's SDK were to make IPP2 RAW processing a single, monolithic function call.

I don't know what the function interfaces of the IPP2 SDK are, but I'm 99.99% sure they expose the stages separately, that there is no penalty for calling each at the appropriate time, and it is up to the NLE vendors to refactor their code so that each stage is done in the appropriate place of the pipeline, not all at once.

 

[Steven Abrams]

At the risk of saying something completely uninformed, but based on years and years in the software industry and an author of major tools for writing SDKs, I will say this about that. RED has published that IPP2 has three stages of image processing. They have indicated that there are multiple things that happen at each stage. I would therefore expect that each stage has its own first-class interface that effectively forces a round-trip between the user and the SDK

I just noticed you have already discovered that in fact, red has not done this and instead made it difficult for developers to implement the sdk as the intended use would imply.

OK...so I did take a look. And to my surprise and chagrin, it looks like RED hasn't made it easy for the NLEs at all. They have made the IPP2 process a completely monolithic function, which sucks for anybody that wants to do grading operations between Stage 2 and Stage 3.

RED, you can do better!

Agreed. As I assumed, the issue is how red have provided the sdk. I imagine phil's post alludes to a change in the sdk coming to address this, but even still this will require quite a bit of work for each NLE to adapt their applications just for a single camera manufacturer so although it may come it may take some time and it will be interesting to see how all the different apps would implement this so that the workflow is consistent across apps.

 

[Hrvoje Simic]

The other day I came across the abomination known as "Shaper LUTs" (pages 870 and 889). This is an idea of using 1D LUTs (which can have lots of 1D precision) to pre-condition data so that 3D LUTs don't clip too badly in their first or last buckets.

Or you can just use a shaping 3DLUT which does not clip at all.

To that I say: this is why CDLs exist! The CDL provides a mathematical transfer function--which is infinitely more precise--than a 1D shaper LUT.

And inferior in particularity compared to a 3DLUT.

 

[Phil Holland]

Jist chiming in relating to the IPP2 SDK. There's a host of reasons it hasn't been the instant coffee version of an update for many 3rd party vendors.

With breaking NDAs I can say I'm testing more than one package with IPP2 support "done right" that's incoming somewhat soon.

When you take into account the great deal of complexity and QC required to tackle things like GPU acceleration, potentially 3 different types if it, CPU or software driven techniques, the general amount of resources dedicated to implementing new things, and the massive kist of other things that need to be vetted before a public release you get a bit of a clearer picture that we are dealing with a lot of moving parts. There's additional levels of complexity when we are talking about some udpates that are more bug fix oriented rather than implementing new features.

That all said, it's coming.

And on a side note I will be pushing two vendors in particular regarding some other stuff. One of which is getting an entry level license down in price so we have more choices on the front that most people (big or small budgets) can utilize serious tools that provide support for faster development on their side and on ours. The other vendor has access to tremendous potential already, but they are wrangling a lot of cats. I think moving forward there is the concept of a well executed 3-tier plan within a development cycle to produce a killer app that adheres to where we are and where we are going. It's just a matter of tranfsforming the abstract into a deliberate path.

 

[rand thompson]

Thanks very much for the Update Phil!!

 

[Steven Abrams]

To that I say: this is why CDLs exist! The CDL provides a mathematical transfer function--which is infinitely more precise--than a 1D shaper LUT.!

And inferior in particularity compared to a 3DLUT.

Most of Resolves 1D shaper luts are 4096 in size - that is going to give you far more precision for straight gamma mapping than any typical/practical 3DLUT. I've seen apps that instead of performing a complex curve calculation on every pixel will instead pre-calculate a 1D LUT on CPU and pass it as a 1k texture to the GPU - no one notices because the precision is high enough! Granted shaper luts can be low precision, but they don't have to be. A 1D shaper lut CAN have more precision than you need and totally depends on the size and what it's doing and what the person who made it decided.

 

[Hrvoje Simic]

Most of Resolves 1D shaper luts are 4096 in size - that is going to give you far more precision for straight gamma mapping than any typical/practical 3DLUT.

And the choice depends on what is the purpose because 1D with else shaped manually with available operators and principles of colour spaces won't achieve the same result as specifically shaped 3DLUT.

And what may help with a complex multi-leveled VFX pipeline may be a setback in grading.

 

[Steven Abrams]

And the choice depends on what is the purpose because 1D with else shaped manually with available operators and principles of colour spaces won't achieve the same result as specifically shaped 3DLUT.

A shaper LUTs purpose is not about changing color gamuts, its about gamma mapping. And for that purpose a 12bit 4096 shaper LUT is a much better choice in terms of precision than a 3DLUT (more than 100 times compared to a 33^3). A 4096 shaper is also only around 160KB where a 33^3 3DLUT can be 1.4MB (depending on decimal places). For linear conversions a 3DLUT is just not accurate enough, which is why shaper luts are used (where math operations are not practical).

 

[Michael Tiemann]

A shaper LUTs purpose is not about changing color gamuts, its about gamma mapping. And for that purpose a 12bit 4096 shaper LUT is a much better choice in terms of precision than a 3DLUT (more than 100 times compared to a 33^3). A 4096 shaper is also only around 160KB where a 33^3 3DLUT can be 1.4MB (depending on decimal places). For linear conversions a 3DLUT is just not accurate enough, which is why shaper luts are used (where math operations are not practical).

Math operations are certainly not practical when you live in a world that denies the existence of CDLs. However, in a software product that does ostensibly support CDLs, it should be moderately preferable to have a workflow that need only attach 3 32-bit numbers (Slope, Offset, and Power) to a gamma adjustment, rather than lugging around 160KB of approximations. Or at least it would be preferable if the software that supported CDLs did so in a way that made them as easy to use and specify as LUTs, rather than treating them as if they were made of Vibranium. Or worse, Kryptonite.

 

[Hrvoje Simic]

A shaper LUTs purpose is not about changing color gamuts, its about gamma mapping. And for that purpose a 12bit 4096 shaper LUT is a much better choice in terms of precision than a 3DLUT (more than 100 times compared to a 33^3).

A 4096 shaper is also only around 160KB where a 33^3 3DLUT can be 1.4MB (depending on decimal places). For linear conversions a 3DLUT is just not accurate enough, which is why shaper luts are used (where math operations are not practical).

Which is why I wrote that it depends on the purpose and goal. 100x higher precision than 33^3 is irrelevant for grading. 64^3 reaches limits of discerning ability. LUT quality and optimization for input signal is more important from that level of precision because it defines image parameter stability when the signal is put through operators, and inappropriate gamma mapping may be a setback regardless of its precision.

I've done heavy grades with 33^3 as a basis and it works. Because I don't use just any 3DLUT which someone made in some way based on something. I mostly use 64^3. Knowing exactly what they do to the signal so I don't have to worry about unwanted surprises and issues in the signal chain. The precision you are talking about is more important for specific multi-level VFX post and cumulative computative losses.

Gamma mapping LUT alone used for creative shaping basis can be a setback for grading compared to 3DLUT because manipulating the rest of the image aspects will be predetermined with operators and color space. There are scenarios where a 3DLUT treating all the signal aspects together is a better starting point. It all depends on a creative approach and workflow priorities but there is no free lunch just because gamma mapping is more precise.