DPX or RAW

[Christoffer Glans]

I've been grading in RAW ever since I started using Resolve and it works nice. But there are different algorithms that different software use to do the debayer of R3D files and I'm curious to know the quality difference between let's say, rendering out to DPXs from Redcine-X before grading in Resolve and grading RAW directly in Resolve. Another aspect could be when working with RAW files in Premiere and After Effects, rendering out to 4K DPX before grading in Resolve.
Also, when finishing in Resolve, is it preferable when doing work in After Effects to work with Redlog film that you later grade or grade before doing work with After Effects?

What's the best workflow for quality?

 

[Bruce Johnson]

All software packages that handle R3D files (including REDCine X) use RED's SDK for reading the R3D file and debayering it. All R3D debayer operations are handled by the RED sdk. You have some control over debayer when you choose the quality setting or use A.D.D. and its associated settings.

The only choice the various software packages have is whether to debayer using the GPU or CPU. Some feel that choosing Full-Res/Premium using the CPU renders a more pleasing, organic image. I'm not sure where the RED Rocket works in the R3D debayer pipeline if at all.

 

[Gunleik Groven]

I'm kinda tempted to go back to DPX grading, and sometimes do.
Doing a nice development in RCX and the grade in resolve makes for less hassle sometimes.
And the debayering eats so much of resolves capacity, that it can feel like releasing the Kraken to go back to DPX grading.

But I should maybe not say that loud

 

[Roger Singh]

Instead of creating DPXs to grade in Resolve, you can just set the Rendering Cache to User, and turn on Render Cache Source Image.

From there, you can have resolve automatically create DPXs for you to save some GPU/CPU resources. The benefit of this option too is that it'll automatically re-create new DPX files if you change the RAW settings within Resolve.

To save even more resources, you can make ProRes 444/DNxHR 444 files instead of DPX for the source Render Cache.

 

[PatrickFaith]

David Fincher's "Gone Girl" has been written up in detail, uses R3D but also is fully integrated with returns from after effects. There's a bunch written up on it all over the place giving the details. Example:
http://blogs.adobe.com/premierepro/...duction-pipeline-for-gone-girl.html#more-2844
http://blogs.adobe.com/premierepro/...orkflow-for-finchers-gone-girl.html#more-2837
http://blogs.adobe.com/premierepro/...ther-milestone-for-adobe-premiere-pro-cc.html

 

[jake blackstone]

David Fincher's "Gone Girl" has been written up in detail, uses R3D but also is fully integrated with returns from after effects. There's a bunch written up on it all over the place giving the details. Example:
http://blogs.adobe.com/premierepro/...duction-pipeline-for-gone-girl.html#more-2844
http://blogs.adobe.com/premierepro/...orkflow-for-finchers-gone-girl.html#more-2837
http://blogs.adobe.com/premierepro/...ther-milestone-for-adobe-premiere-pro-cc.html

In case of "Gone girl", the conformed movie was delivered to LI for grading in DPX form. Grading from RAW is overrated and often unnecessary, especially where VFX are used...

 

[Marc Wielage]

In case of "Gone girl", the conformed movie was delivered to LI for grading in DPX form. Grading from RAW is overrated and often unnecessary, especially where VFX are used...

Jake is very wise. :iagree: DPX's or even ProRes can work fine provided the transcodes are done properly.

 

[Gunleik Groven]

Jake is very wise. :iagree: DPX's or even ProRes can work fine provided the transcodes are done properly.

And leaves your system to do the task you want: Manipulate colors.

There are a ton of (potential) advantages to go to RGB, especially on larger projects.

I really dig/love RAW for it's flexibility. But that doesn't mean that I always have to work on a RAW file at any time for any operation...

When grading from RAW, you do grade a RGB signal. And you spend lot of horsepower on getting that signal consistent.

 

[Peter Lyons Collister ASC]

For my current feature all r3ds were transcoded to Open EXR and the grading will be done using those transcodes

 

[Christoffer Glans]

For my current feature all r3ds were transcoded to Open EXR and the grading will be done using those transcodes

Yeah, that would be the best way to do it, but it's pretty heavy to work in that format, especially if you want to go with 6K or 5K files in order to reframe or stabilize shots. It might work on the real high end productions where that kind of budget for post isn't an issue.

Wouldn't it be best if you could do the "online" before grading? What I mean is that you edit offline and conform the timeline to the R3D RAW files and then do reframing, stabilizing and VFX in Open EXR that you then send as 4K files to grading? That way you don't have to work with 5-8K files for a 4K master but instead use the exakt resolution that the master is going to be in.

The main reason I'm always hitting my head on is how to handle VFX shots. Doing something that's all in camera is very easy to grade and master, just RAW in and a master out. But as soon as you need to fiddle with the image you need to transcode to another format in order to work with it, either do the grade first and then the VFX or do the VFX before grade (even though my tests seem to show that working in that way degrades some of the depth compared to the original files when applying grades). So maybe it's better to just do the "online" VFX, stabilization etc. in Premiere and After Effects (for those who use these) and then go to Resolve for a final grade as DPXs.

Then comes the questions of a LUT applied before DPXs. In order to maximize DR before grading it seems wise to put some gamma curve on the footage so that it isn't entirely LOG-mode. My hypothesis, which might be wrong, is that without some gamma curve the footage will be "compressed" in some way compared to the RAW files. So in order to maximize what you get from the RAW files you need to apply "something" as it renders to DPX or other formats. That way when you grade you have a little more information in the image.

Thoughts?

 

[Gunleik Groven]

If you lower the blacks in say RLF, you ditch precision in the blacks.
You really don't maximize DR that way.

The default in RCX lifts the blacks a bit, which I used to do also when batching shows for grading (after a bit of A/B testing with the color people)

Linear EXR's are great, BTW. But used to be more cumbersome to work with.
I'll go there again shortly after reading Graemes thoughts in the ACES chat under workflows. (I made that a sticky).

 

[Stacey Spears]

Peter, are they using compression with the EXR files?

 

[jake blackstone]

When grading from RAW, you do grade a RGB signal.

And when grading from DPX or Prores you don't?

 

[Gunleik Groven]

And when grading from DPX or Prores you don't?

Sorry for the lack of precision.
Of course you do. But a few people seem to think that the signal you grade on when graded from undebayered RAW somehow isn't RGB... And has some magic powers/dynamic range/colors outside the transform you apply to it post debayering and a LUT (as the matrix seems to be hardencoded, if nowhere else, at least in the SDK) over the same signal treated the same way and "baked" into say a cineon log/RLF 16 bit dpx, or a linear 16 bit+ EXR at the same target resolution


My point was that sometimes for a multitude of reasons, it can be a huge timesaver and consistency-security-tool, to actually make those RGB files before approaching DI.
And as to computing powers, if you don't spend most of your GPU debayering, you can have a bit more fun with your secondaries

But I know you knew I knew that...

For the point of avoiding hairsplitting, let's not discuss prores, which I like a lot.

 

[Christopher*Seguine]

The main reason I'm always hitting my head on is how to handle VFX shots.

EXR is best for VFX. Most houses are based on and EXR workflow.
Resolve now supports multichannel EXRs, well mostly.
So you send out your source plates out as EXR, you get back a multichannel EXR comp - giving you individual layer control for grading.
Sure you could do it before with external mattes, but much cooler that its all in one file.

R3D vs EXR - your trading needing needing fast CPUs vs needing fast storage, both work, choose where you want to throw your money.
Main advantage of staying raw is re-framing, if your shooting 6K FF to deliver 4K DCI you have plenty of room to work.
Much easier to work with a small 6k R3d, 6K EXRs for anything besides VFX selects becomes a huge burden for a feature very fast.

 

[Christoffer Glans]

If you lower the blacks in say RLF, you ditch precision in the blacks.
You really dn't maximize DR that way.

The default in RCX lifts the blacks a bit, which I used to do also when batching shows for grading (after a bit of A/B testing with the color people)

Linear EXR's are great, BTW. But used to be more cumbersome to work with.
I'll go there again shortly after reading Graemes thoughts in the ACES chat under workflows. (I made that a sticky).

I'm not talking about putting a mega-gamma on it, not even the Red Gamma, I mean a slight gamma so that it isn't as compressed as RedLOG film is. As long as the gamma curve doesn't clip or crush the highlights and shadows you still have everything there. The thing is that even if you grade in RGB the RAW-file is handled as RAW in the first node, which means that if you change the RAW-settings you pull out more information from the RAW-data. If you render to DPX first you loose that data and instead use RGB all the way. So in order to get the most out of RAW as possible before grading in RGB, a slight gamma curve to reduce the extreme washness of redLOG film could provide a little more information for your starting point.

EXR is best for VFX. Most houses are based on and EXR workflow.
Resolve now supports multichannel EXRs, well mostly.
So you send out your source plates out as EXR, you get back a multichannel EXR comp - giving you individual layer control for grading.
Sure you could do it before with external mattes, but much cooler that its all in one file.

R3D vs EXR - your trading needing needing fast CPUs vs needing fast storage, both work, choose where you want to throw your money.
Main advantage of staying raw is re-framing, if your shooting 6K FF to deliver 4K DCI you have plenty of room to work.
Much easier to work with a small 6k R3d, 6K EXRs for anything besides VFX selects becomes a huge burden for a feature very fast.

Yeah, that's what I meant by doing an offline edit and then relink to the R3D files to do the resize and stabilization before rendering out to either DPX or open EXR in 4K. In that way you get good debayer down to 4K, although you might want to go 5K for VFX if it demands some manipulating with the image or needs some safe outside the frame for tracking.

As for the EXR workflow in Resolve, I've never tried it. What's the step by step for doing it correctly? Render out to EXR in Redcine, then import into Resolve? I presume that the EXR workflow is for getting an ACES master in the end? If so, how do I setup my resolve for ACES in order to do a proper grading with my video calibrated monitor?

 

[Peter Lyons Collister ASC]

For my film the transcodes were to 2K Open EXR but no compression. If there is reframing to be done or something special then they go back to the reds. WETA is doing all the VFX work and if they were to work with either 4K Open EXR or the original reds the cost per shot would greatly increase. As rendering time and storage continues to decrease all VFX work will be in 4K.

But believe me when the VFX budget is 50 million you don't want to add millions just to get the VFX in a 4K world.

 

[jake blackstone]

The thing is that even if you grade in RGB the RAW-file is handled as RAW in the first node, which means that if you change the RAW-settings you pull out more information from the RAW-data. If you render to DPX first you loose that data and instead use RGB all the way.

Is it something in the air today or what?
For the billionth time- you can't grade RAW!!!
Every software does the same thing.
1. It decompresses compressed R3D.
2. It debayers it, i.e turns bayered image into linear RGB image.
3. If needed, it maps Linear Gamma into a Video or Log gamma or it is left as a Linear image, which is used for creating an EXR.
4. It may encode into something like DPX (Nucoda), Uncompressed (Resolve or Baselight), EXR for ACES for all grading software or it's variation, like the new Truelight color Spaces for Baselight.
And only then you can apply the first color node or layer, depending on software used.
All this happens in real time, so you may THINK you're grading RAW, but in reality you're ALWAYS grading transcoded RGB image.
The only time you're "grading" RAW is by adjusting the metadata, like FLUT or Color temperature, Tint etc.
Please stop it with FUD. By properly using full RGB encoded uncompressed files for grading you lose NOTHING for reasons I just explained...

 

[Stacey Spears]

For my film the transcodes were to 2K Open EXR but no compression.

Thank you Peter.

 

[PatrickFaith]

Is it something in the air today or what?
For the billionth time- you can't grade RAW!!!
Every software does the same thing.
1. It decompresses compressed R3D.
2. It debayers it, i.e turns bayered image into linear RGB image.
3. If needed, it maps Linear Gamma into a Video or Log gamma or it is left as a Linear image, which is used for creating an EXR.
4. It may encode into something like DPX (Nucoda), Uncompressed (Resolve or Baselight), EXR for ACES for all grading software or it's variation, like the new Truelight color Spaces for Baselight.
And only then you can apply the first color node or layer, depending on software used.
All this happens in real time, so you may THINK you're grading RAW, but in reality you're ALWAYS grading transcoded RGB image.
The only time you're "grading" RAW is by adjusting the metadata, like FLUT or Color temperature, Tint etc.
Please stop it with FUD. By properly using full RGB encoded uncompressed files for grading you lose NOTHING for reasons I just explained...

+1, also how I explain it to some people is that you loose the r3d no matter what after you do the compositing, so it's impossible to grade to raw post composite. (i.e. no program can "write" a new R3D). Now some people don't want to go through compositing, I say good luck to that. Another thing is almost everything is max 12 bits after it goes through compositing, and is easy 10 bits in log ... so the only reason to keep more then 10 bits log is if the compositors went outside the color gamut (above 1.0 rgb values I view as screw ups where people didn't linearly add with alpha multiplies during compositing).

I btw would like to look over someones shoulder to understand true light color spaces for baselight.