RED workflow in post - not a step forward? a serious discussion

[Louis Eguchi]

I am at Bournemouth university studying computer animation. often we have "master-classes" were representatives and professionals from the industry deliver a lecture on their respective practices and experience. we have had visits from The foundry, dreamworks, Double negative, prime focus and more!

recently we had a talk from prime focuses R&D and pipeline engineers - these guys have worked on many blockbusters. I am like many of you a big fan of red and the innovation of these fine Cameras. and thus decided to blow my summer earnings on shooting my final degrees major project on RED (Red one) at 4k. this film is a VFX and features effects done in NUKE and Houdini.

Personally on the post side i have undertaken compositing and editing and have gained some experience of the work-flow with only the internet and limited experience as a training DIT.

soooo...

I have encountered many learning experiences with my much loved R3D files through the post pipeline and decided to ask the pipeline engineers a couple of questions regarding Working with RED cameras, seeing as it seemed the perfect opportunity to get some professional advice on the matter. i have to say i was disappointed.

I asked the guys - "i have some 4k red footage, and am using it in NUKE for my VFX film, i have had to learn a lot to optimise my work-flow, while trying as hard as possible to retain the RAW capabilities and resolution while working on modest workstations - how do you guys deal with RED footage"

to which they replied - "do a preliminary grade then render out HD tiffs work with those, RAW files are too expensive to work with."

The reason i was disappointed is that i want to be able to use the raw files and resolution right through my compositing pipeline, i want to grade during my nuke scripts and use the parameters (which you can do to an extent with the read node.) to me it seems an incredible waste to go to all the effort of having a red camera and then doing away with all advantages and render them as standard TIFFS or DPX's is RED not supposed to be glorious for the post pipeline?

maybe i am being silly here, maybe it is the 'best way' but i long to be able to utilise the the RAW codec right through my pipeline. i made some propritry tools in python to help my workflow import looks etc stuff that doesnt yet exist, i would love to have the time to develop a tool that grades the RAW files somewere down the line in my nuke script.

I want to utilise red in the best way possible and not take a step backwards.


Best Louis Eguchi.

 

[Bruce Allen]

The guys from Prime Focus know what they're doing.

DPX files or EXR files are the norm because it often takes too much computing power to decode the damn R3Ds at full quality.

It depends what you're doing though - some workflows would be awesome. But trying to composite a bunch of layers of R3Ds would kill the average artist's workstation pretty quickly.

And yes, 5K is great but sometimes it makes sense to scale down or use proxies. Especially since it's a Bayer sensor so it's a bit of a waste to be calculating for every R,G,B element at 4K, when in actual fact the camera doesn't contain true R,G,B detail per pixel, since it either captured R or G or B.

If you did a full-quality debayer, got the white balance right, and scaled the 4K stuff down to about a 3.2K EXR file, you'd be good. Nuke works with EXR files as its native format so it's super fast with them.

BTW:
Film has greater range than RED, unless you use HDR mode. And DPX files worked well for film. Sometimes you'd have to re-scan but not often.

An EXR file can capture many, many stops of latitude more than RED.

And if you use wide color primaries (see ACES discussion, but also P3), you can describe all the colors RED can capture and more.

Just get your white balance decent, and consistent shot-to-shot!

I can guarantee you that all VFX-heavy feature films throw any ideas of "staying RAW until the end" out the window for this (and many other) reasons. Read the workflow stuff for Dragon Tattoo.

Bruce Allen
www.boacinema.com

 

[Louis Eguchi]

Many thanks for your reply - i am definatly ignorant to think that prime focus are missing something, your explanation of these image formats rule out any of my worry for loss of information. I often find it frustrating when the post world seems to have so many work-flows, codecs and formats its hard to know which way to go.

may i ask why you suggest 3.2k? is this because it is a good compromise resolution to work with?.

do you also suggest to get the white balence consistent before opening a nuke script because this will best hold the integrity of the data, or just good practice for continuity?

may i also ask were to find the dragon tattoo work-flow article?

Thanks again

 

[Erich Ocean]

may i ask why you suggest 3.2k? is this because it is a good compromise resolution to work with?

At 4K capture, that's all the resolution that the debayered output actually has. If you output any higher, you're effectively upscaling from that 3.2K.

With 5K capture, you can get a clean 4K debayer.

do you also suggest to get the white balence consistent before opening a nuke script because this will best hold the integrity of the data, or just good practice for continuity?

Both. Red's color science requires that you set the white balance correctly before debayering. It's also good practice to get this (very simple) step correct right from the start.

To add to Bruce's list:

1. set the white balance
2. colors to RedColor3
3. output to RedLogFilm
4. save to DPX or EXR (like Bruce, I prefer EXR and since you're in Nuke, definitely do it). You can use compressed EXRs to save space.

Best, Erich

 

[Nick Shaw]

Nuke can use a Rocket card if one is available in your system, but because the Rocket SDK is optimised for real time playback, which is not how Nuke works, there is not as much of a performance gain as one might expect.

I concur with the idea of rendering out down-sampled EXRs or REDlogFilm 16 bit DPX files for a VFX pipeline. You can always keep the R3D files available to the system so you can go back to the RAW for a particular shot if required.

 

[Nick Shaw]

Another argument against using R3Ds in Nuke is that Nuke (currently, as of 6.3v7) does not have the option of REDcolor3. The improvements in colour science with REDcolor3 are worth having IMHO.

One extra suggestion is that if you do use R3D files in Nuke I recommend selecting "Half Float Linear" as the "gamma curve" in the R3D section of the Read node rather than the default "Linear". They are very different, as explained in the Nuke contextual help.

 

[Louis Eguchi]

This is another problem i am having with the workflow in nuke is the color. i am in nuke 6.1v3, when i do a grade in RED CINEX pro i am using the default color space REDcolor and the REDgamma curve. when i am in Nuke the read node defaults to decode color space - REDcolor and the gamma curve is - linear. this is very wrong in comparison, i understand nuke works in merits with its linear color space arithmetic , but this looks wrong. to get the closest representation i am setting the standard read nodes color space (the one at the top before any of the R3D specific parameters) to sRGB. and the decode color space to REDcolor and the gamma curve to REDgamma. this is very close but not right.

in terms of the project my scripts are working in 720p tiffs to be able to stay sane when working with large scripts that are easilly adapted to use the 4K R3D's. sorry to anounce these amateur issues i know its common for people to complain about how their footage looks different when doing the ''app dance"

perhaps these questions have been answered elsewhere or should be in a new thread.

many thanks for the explanations of the debayer and effective resolutions.

 

[Nick Shaw]

If you use sRGB/REDgamma you are not making best use of Nuke's unclamped float pipeline, and you are compressing your highlights with the REDgamma curve, so potentially losing highlight detail.

I don't remember what options are available in Nuke 6.1v3 off the top of my head, but I don't believe "Half Float Linear" was an option. I would suggest REDcolor2 (if available) rather than REDcolor, although you will then probably need to boost saturation later with another node. Then set "colorspace" in the Read node to "Cineon" and "gamma curve" to "REDlogFilm".

But again I reiterate that you are making your system work unnecessarily hard decoding every frame from the R3D. Test a REDCINE-X -> EXR -> Nuke workflow.

 

[Blair S. Paulsen]

Options are good, if you choose wisely...

Options are good, if you choose wisely...

It is my belief that within a year all the major applications we work with will handle R3Ds natively from the users perspective with rendering left for final output. Internally it may take a 1/4 extraction from the wavelet and create an RGB proxy, but assuming you have a decent workstation with a newer CPU and plenty of RAM any slowdown should be minimal.

Nick and Bruce have both offered advice I suggest you follow. I am about to start a new long form project and we are looking at a compressed OpenEXR solution to facilitate management of disparate elements. We are planning to transcode everything using RC3/RLF. First light grade to set WB, mid-gray, and on some shots to take the image part way toward the intended look so that we have enough information in the right places for the final grading pass. YMMV.

Bottom line - I always look at a project as a whole, and choose the most effective strategy for that circumstance. For instance, if most of your VFX shots are in just 2 or 3 scenes of a long form piece I might not go the DPX/EXR route on the whole show right up front, etc.

Cheers - #19

 

[Gunleik Groven]

http://www.reduser.net/forum/showthread.php?78453-RAW-workflow-a-discussion

 

[Marc Wielage]

It is my belief that within a year all the major applications we work with will handle R3Ds natively from the users perspective with rendering left for final output.

They could, but: what if you're dealing with an effects-heavy project with eight different VFX vendors? How do you deal with multiple layers of composites, all in Raw? What about files that are completely generated digitally, from scratch? What if you have a Red Raw foreground image, and a completely CG background? What if the Red footage is shot badly, or inconsistently, or things go horribly wrong? What if multiple cameras are used? What do you do with the R3Ds then?

Sure, you can use R3Ds for straight-ahead live-action projects with minimal effects, but I agree 100% with what Bruce says above. I think for a real workflow where time is of the essence, and you have to conform, color-correct, and output material with limitations on time and budget, DPX is a wise choice. In the real world, you'll still have lots and lots of range to work with in terms of color and exposure.

The reference to Dragon Tattoo is very apt. That workflow obviously worked very effectively, and proved that there are many ways to get to the finish line. From my perspective, too many people are getting wrapped up with theoretical methods and not considering the most direct, cost-effective approach. I have no problem with keeping everything 4K all the way, provided you have a very fast pipeline, tons of storage, and a considerable budget. In the real world, if you have a small film with a modest distributor that's mostly going to be seen on Blu-ray, download, and cable, I'm not sure it's going to be practical for indie filmmakers. Big studio projects, no problem.

And I still have many issues with long-term archival storage. But that's another topic.