Color or Visual effects, which comes first in the workflow

[Will Turner]

I've always been curious about which comes first in the post workflow process. Color grading or visual effects? A lot of times I like having the raw meta data when color grading to make adjustments on that base level, yet once rendered doing some visual effects like keying or roto work become much more difficult with a look baked in. What do most post workflows look like?

 

[Serge Tsipis]

I'm not that much into VFX like some redusers here. But when I have to, grading comes first. I guess for two reasons. First is that graded look may be somewhat different and it is important to take it in account doing VFX. Second is that R3D is not really compatible with all VFX software I use. So mostly I do RCX RMD creation with minor adjustments (WB and curves) - edit in FCPX - roundtrip to Resolve - export to highest quality ProRes - roto/key/composite all VFX tricks - reimport to FCPX for assembly and finalisation.

My toolbox is: FCPX, Resolve, RCX, Mocha Pro, Boujou, Cinema4D, Nuke, Motion.

I look forward for other more experienced opinions to arise on this thread. Will be glad to learn different approaches and hear from experienced VFX guys.

 

[Phil Holland]

I've been doing VFX related work for a while. There isn't really one answer, but what's common on larger budget features is you deliver plates to the VFX house quickly so they can get started with comps or tracking. Then as color is approved you deliver updated plates with the correct CC. For studios that do it all in house or houses working with each other this is pretty common.

If a shot requires something like stabilization or a crop that can happen in color or in comp, just depends on the workflow.

While having the .R3Ds is rather important for color work, what I deliver to studios is often 16 bit DPX, TIF, or EXR depending on their needs. Some studios have proprietary file formats as well that get used, though that's pretty rare these days. When gets shot out via laser recorder or delivered is the image sequences to be assembled into a DCP. In broadcast things like ProRes sequences come into play and those get created by the post house usually.

 

[Paul Sage]

Like Phil said, not really one answer. Typically in features the vfx are done first or on top of pre-graded plates (70% color or so, not full). In commercials, I have seen VFX done last on a good amount of projects, after color is finished. Really just depends on the schedule in most cases.

yet once rendered doing some visual effects like keying or roto work become much more difficult with a look baked in.

If your have an issue pulling a chroma key on a graded plate, you can pull the key from your un-graded plate and then use that as a matte in your graded plate comp.

 

[Marc Wielage]

I think Phil and Paul above both have reasonable answers. The real issue I've encountered with studio pictures is that the VFX houses fall behind and the work starts getting farmed out to 4 or 5 (or more) different houses, and the work starts coming back in with the color all over the place. I've had some screaming arguments with VFX people who insist that "somebody" approved a plate, and when we drop it into the finished show, it looks like it's from the wrong movie. The issue quickly becomes how much you can fix with the time available, and that can be very problematic.

I'd vote for giving the VFX house reference plates so you can say "here's the direction we're going for the final," and also agree on LUTs and color management so that everybody is seeing more or less similar pictures in different facilities. Ideally, the post supervisor should be involved with this (as well as the VFX supervisor and VFX editor). The VES Handbook of Visual Effects goes into a lot of "best practices" on how to handle workflow with features and TV series.

I can tell you straight up that I haven't seen any VFX houses use Raw files, and in fact, VFX-complex projects traditionally use DPX log files. Pretty much every summer blockbuster movie you've ever seen was done that way (with rare exceptions). Of course, it's important to do tests beforehand and make sure the Log images will provide the colorist with what he or she needs for the final. I find using RedLogFilm works fine for what I need, except in extreme situations where something's gone very wrong.

 

[John Brennick]

In my experience different shops work differently. In comp we're either provided a neutral grade for the plate thats or we are given a lookup/grade that mimics what theyre going to do in the DI that can be reversed at the end of the comp. I've never been given a look directly baked into the footage, especially with greenscreen elements as you want to preserve the integrity of the footage as much as possible. Once we start the shot, either a neutral grade or the base color grade is applied to the plate. The lighting department tonemaps the hdri's to the plates we have our base grade (or neutral grades) applied to and they build their lighting rigs from there. The lookup that is provided to us as a base look is put into a reversible gradenode which can then be re-applied and reversed out at the end of the comp keeping the plate in tact with how it was delivered to us. The client is then delivered the graded and undgraded versions to do what they wish. In Nuke you can take your grade and group it and name it VIEWER_INPUT so no matter what youre looking at, you can see how the grade will effect the element. This is all just to reaffirm that there really isnt one "correct" workflow. Different strokes for different folks I suppose.

 

[M Most]

I'd vote for giving the VFX house reference plates so you can say "here's the direction we're going for the final," and also agree on LUTs and color management so that everybody is seeing more or less similar pictures in different facilities. .......

I can tell you straight up that I haven't seen any VFX houses use Raw files, and in fact, VFX-complex projects traditionally use DPX log files. Pretty much every summer blockbuster movie you've ever seen was done that way (with rare exceptions). ....

The most common scenario on large VFX pictures today (and we do a LOT of front end work on those kind of projects) is to send the VFX vendors image sequences (often DPX but currently moving towards Open EXR) along with an output LUT and CDL values from the dailies processing. That way, everyone gets the same initial result as the "approved" dailies. That's why CDL's are considered very important in large feature workflows, as they are put in the ALE and get stored in the Avid bins and sometimes the MXF files themselves, so they can easily be recovered. In fact, I would say that the two most common uses for CDLs are for moving on-set color settings to either the near set or facility based dailies system, and for supplying VFX vendors with a method to restore the same color settings as the dailies in a "soft" manner. On a Red project, that would usually mean processing everything using the same color space and gamma (most commonly Redcolor3 and RedlogFilm), and using a CDL and output LUT on top of that throughout the process. If the project was only shooting Red (I've yet to see that, but it probably happens sometimes) you could in theory send trimmed R3D files and the RMD, but that is not a common scenario, and I've never seen it done on a big picture.

 

[Elliott Balsley]

Secondaries

Secondaries

This is a good thread.
As a DIT, I sometimes find myself wishing to do secondaries for the dailies, when there is some obvious problem I can fix quickly, like a blown window. But of course, this grade wouldn't translate with a CDL. What do others do about this? I've heard the argument that the DIT should never do secondaries, period. Currently I'm making the fix, as long as the DP is aware that it won't match later.
Sometimes, if they are planning to use Resolve for final color, I've sent the whole DRP project with all my looks, but that doesn't really help with VFX. It would be great to have CDL v2.0.

 

[Paul Schneider]

Most places I work with generally do vfx on the ungraded shots and use a viewing LUT within the edit and composite to get an idea of what its ultimately gonna look like. The color session, final conform and mix then usually comes at the end.

 

[Marc Wielage]

As a DIT, I sometimes find myself wishing to do secondaries for the dailies, when there is some obvious problem I can fix quickly, like a blown window. But of course, this grade wouldn't translate with a CDL. What do others do about this? I've heard the argument that the DIT should never do secondaries, period. Currently I'm making the fix, as long as the DP is aware that it won't match later.

I think it's fine for dailies, but in truth, the DP needs to know where the problem spots are going to be. What I've seen done for the editor for problematic shots is for two versions being prepared: one just straight, and the second a windowed correction fixing the problem with notes placed in the EDL. The final colorist is going to be aware of it when they get knocked out of their chairs when the original material pops up.

I don't see the CDL ever being able to handle windows, keys, secondaries -- it's just too much to expect. As it is, I would rather the CDL address issues like dynamic color changes, which are necessary to handle exposure changes mid-shot. And we still don't have that, either, and it's still a very straightforward slope/offset/power correction that could be described with numbers.

It's unfortunate to reflect that the DP should be trying to fix something like a blown window with filters, rather than trying to worry about fixing it in post with dailies. I grasp that there's only so much time and money, but to me, you have to scale your expectations down with the situation, knowing full well it can be fixed later on. On the other hand, I can vividly remember many dailies sessions of the 1990s and early 2000s where we were doing all kinds of tracking windows and secondaries, knowing what the DP expected for dailies. This wasn't typical, but it did happen sometimes, particularly for dual-path (corrected/uncorrected) film dailies. I see no reason why something similar couldn't be done at a DIT station, given lots and lots of time and money.

 

[Peter Majtan]

From the days way back when used to work as a VFX supervisor for projects shots on 35mm - the workflow was simple. We would always get the scans either RAW, or with 1st light applied - never the final colour. One of the main reasons for that was that the final grading process helped a lot in "blending" the VFX in making them all but invisible to the eye...
These days ironically I am on the DP/Color duty and always outsource the VFX. Even so - I always give the VFX house the RAW files. The most common VFX SW use (and "prefer") RAW data, such as NUKE. The CFX guys simply match the effect to the basic look of the RAW files. I get back comped TIFFs (or rarely DPXs). In the unlikely scenario where I need the RAW files for colour - I will ask for the VFX to be rendered without the BCKG and I do the comp myself...

:sifone: Peter

 

[Serge Tsipis]

Thanks for your input, guys! That was educating.
I think I gonna change my workflow to use LUTs and achieve more precise control over 2nd grade after composite. At least for music videos.

What you think of a grading when there are clean plates involved for object removal? I mostly use it for digital makeup, architecture/set treatment and wire removal?

 

[Michael Cioni]

The most common scenario on large VFX pictures today (and we do a LOT of front end work on those kind of projects) is to send the VFX vendors image sequences (often DPX but currently moving towards Open EXR) along with an output LUT and CDL values from the dailies processing. That way, everyone gets the same initial result as the "approved" dailies. That's why CDL's are considered very important in large feature workflows, as they are put in the ALE and get stored in the Avid bins and sometimes the MXF files themselves, so they can easily be recovered. In fact, I would say that the two most common uses for CDLs are for moving on-set color settings to either the near set or facility based dailies system, and for supplying VFX vendors with a method to restore the same color settings as the dailies in a "soft" manner. On a Red project, that would usually mean processing everything using the same color space and gamma (most commonly Redcolor3 and RedlogFilm), and using a CDL and output LUT on top of that throughout the process. If the project was only shooting Red (I've yet to see that, but it probably happens sometimes) you could in theory send trimmed R3D files and the RMD, but that is not a common scenario, and I've never seen it done on a big picture..

I find that sometimes following the way the "BIG" shows do things doesn't mean it's the best. There are companies that insist on coloring first and delivering colored plates to VFX. While an argument can be made that plan is easier, an equal argument can be made that it is more compromising.

I agree with Mike's assessment of CDL importance. Projects big and small can use CDL on set and then partner those values with VFX pulls. CDL programs are fairly inexpensive, and if you are doing a planned VFX job with RED, one way to do a simple CDL workflow with RED is to:

1. Switch the camera monitoring path from RG3 to REDLogFilm
2. Route the RLF SDI to your CDL viewing kit on a DIT cart (Pomfort,LinkColor,ExdLive, etc.)
3. Apply a transform LUT to normalize the image first (Cineon works fine as a generic transform from RLF)
4. Apply your custom CDL looks atop the Cineon transform
5. Save the CDL values in a database. Or you can create a little automator script to move the CDLs into the RDM directories if that helps track things

The 2nd option is one that I have a hunch RED will offer very soon: A CDL compliant mode of RCX. This should work as follows:
1. Shoot RG3 as considered "normal" with RED
2. Load clips into RCX and put program into "CDL" mode
3. CDL mode will disable non-CDL compliant values and limit image manipulation to CDL controls using the RG3 transform
4. Export CDLs directly out of RCX with RG3 transform and insert them into each RDM folder (each R3D will have its own .CDL text file)
(you can do this option with other tools, but once available in RCX, I predict it will become the most popular with R3Ds and VFX)

The point of this is that for the best control in color, you always need to deliver uncolored material. Sounds obvious, right? But when it comes to tracking all that and applying it correctly across people you may not be dealing with face-to-face, some people default to pre-graded effects and therefore compromise the colorist at the end of the job, not to mention eliminate any consistency between temp VFX and the original dailies.


There is one more workflow that beats them all, however. The RMD workflow. As Mike mentions, it is technically the best way to go with RED and actually it's one of the only camera systems that can actually use a fully functional end-to-end RAW workflow. But it's rare. This is because an RMD workflow means all VFX vendors need to rely on R3Ds with paired RMDs or you'll lose the ability to match everything and maintain a log output once complete. I do this on both small films and large films, most notably in collaboration with Digital Domain on Ender's Game. Here's basically how that would look as a 3rd option:

1. Shoot RG3 as considered "normal" with RED
2. Load clips into any R3D color tool (including RCX) and adjust the look based on the RAW controls
3. Export a new RMD
4. Deliver R3D Trims when delivering VFX plates to vendors along with the RMD sidecar look
5. Vendors load the R3Ds and work with the RMDs directly (such as in AE or Nuke) or can transform to ExR with the RMD values applied atop a linear to log transform
6. Render out RedLogFilm DPX at the end for matching the other R3Ds in the finishing/coloring

I did an in-depth case-study on this very thing and you can see it here (https://vimeo.com/78581143). The most complex thing about the whole ordeal is getting everyone on the same page. That's the biggest challenge with any technological change, isn't it?


If you can't rely on an RMD workflow, plan for CDLs. If you have an R3D-native workflow in the pipeline, then by all means, use the RMD option and take advantage of what RED RAW can offer in terms of automated color metadata.

Each of the above, when done right, work like a charm.

m

 

[PatrickFaith]

Both Mike & Michael's post should be a sticky. I asked this same question when i started my low budget film, my question was "If all decent looking VFX is done in linear, what is the best way to deliver to a Linear workflow?" Basically I got 3 answers back:
1) Deliver in DPX & use a log to linear node in nuke (or related program)
2) Deliver in EXR & make sure u don't do something stupid before you output in linear (no extra nodes required)
3) Deliver in R3D trim & it is just one button click to move it to linear in nuke (on the read node)
If you don't know why VFX people work in linear, you need to sit down with one and have them slowly work you through examples (basically "log(a+b)" is not equal to "log(a) + log(b)".

 

[Gavin Greenwalt]

My preference and how we handle most commercials is to do all comp and post on the Redlogfilm without any color. We use LUTs to preview the color but if the composite works, it works. Then when it goes to color they can regrade it however they want as if the VFX were just in the original plate. In fact, we just swap out the DPXs from the raw plates with the VFX plates and the grade should remain the same.

The largest problem with doing VFX on graded plates is that you're trying to comp in potentially inconsistent color. If a CG element moves through a power window you have to recreate that effect (and you have no idea what the colorist did precisely). If it's in a RLF plate though the power window just automagically affects both the VFX and the original plate.

 

[Stefan Scherperel]

I just completed the visual effects for the film "The one I love" and the process was similar to what Peter mentioned. I worked both as the VFX supervisor on set and then did all the post work as well. I made sure that I was given all files associated with VFX in .R3D format. I talked with our colorist before hand and was given neutral RAW settings to work with. i then used a viewing LUT during the post process as everything was set to Red logfilm prior to grading. I then sent 16bit DPX files of the final comps to our colorist and he graded them along with the other shots from the RAW files in Resolve. Pretty simple workflow that ended working out great.