RAW workflow - a discussion

[Phil Holland]

Agreed on all points. Mostly. As to editing, I still think that depends on the environment one is working (not software).

As to colors and VFX, I am a bit hung. But as long as you ensure consistency, that's all fine, one way or the other.

Point that you obviously understand - comming from film - that seems to be overlooked a bit by a few others, is that the signal we grade actually isn't RAW in any way, and neds the same care for being optimum as it does when transcoded to DPXs. And if you do not posess the machine capacity to do that in realtime at full quality, a transcode might still be a good option for speeding up things, securing consistency and keeping integrity of the images in place.

Exactly. Every job is different and most houses are different. And on top of that there are variances still on whether a colorist wants RedGamma and RedLogFilm depending on how the media is going to be used. We have all of these in the .r3d, which is very nice.

When it comes to the grade I agree, but I still think there's advantages working with native .r3ds in the grading suite or even doing a first light in RC-X Pro and then transcoding. On Baselight, and likely Speedgrade, I do my first and secondaries in the suite on the .r3d It's having the flexibility to make the raw workflow work for you that I like. There isn't just one way to do it. Transcoding can be done whenever you feel it's necessary.

The question is: why is it so easy on RED to create bad RGB files if the R3D development settings aren't right?

Why is setting the white balance right in the RAW file so important?

@ Bruce, as to WB... I Have found that you get fairly undistorted images in the WB realm of 4400 - and 6000. I haven't managed to see any baddies introduced through tint.

The sensor is rated at approx 5000k and I've found through using optical filtration under tungsten to bring it up into the 4400-6000 range that Gunleik mentions you can indeed produce a superior image. Especially where color noise is concerned. That said, without filtration I haven't seen an issue of getting the WB right or wrong in camera (yet). If for some reason I shot 3200k without blue filtration and if it's a low light shot I don't mind doing a bit of noise reduction to the shot.

I notice that native sensor kelvin effects pretty much all CMOS based tech in the same way.

Noise reduction is where the real difference comes into play between Red and other folks. They leave it up to you to get it done. For me this is fine. For many others it's not. I could totally see Dragon having some sort of general on chip reduction, but it's certainly not something that I feel is necessary.

[Blair S. Paulsen]

I think the concept of RedCine "Light" is an interesting one. Mike Most and others have pointed out that the power of RAW is desirable in theory, but in many instances it just makes it far too easy to screw things up. For people like Gunliek and most of us reading a thread like this on "RedUser", the power of RAW would have to be pried from our cold dead fingers, but as Peter Parker learned the hard way...

I would suggest that a version of RedCine (or maybe just a basic/advanced mode toggle) that was as simple as possible might really help the legions of folks who just want a "pretty picture". What if all the user had to do was put an eyedropper target on something white/gray, click on some skin tone in the image if there is any and hit "enhance" or "alchemy" if they want a bit more "pop" and then transcode using a list of pre-sets aimed at the 20 most popular formats.

Cheers - #19

[Gunleik Groven]

I have made a little preset for our colorists getting lowbudget quick turnaround RED shots in from non-drama/promo parts of the house.

RedLogFilm/RC2 (for now)/5000k/640ISO

Works allmost shamefully well for an incredible amount of shots.

Not perfect for every, but "good enough" for more than most. Allmost embarrasingly so...

Even though a proper meta firstlight outshines this, that setting kind of works "very well" for "allmost all" shots out of the box.

And gives the colorists all the RGB fun they don't have with any other camera around here...

As a by-the-by and a bit OT to this thread, we looked at how efficiently Resolve could key out IR pollution from most of the shots we tried it on.
Quick key and desat cleaned up a lot.

This was a bit of a "for dummies" approach, but that can be rather effective.
Stuff doesn't HAVE TO be hard... :)

[Gunleik Groven]

As a BTW. I do firstlighs in RCX-P (whether the shots are to be transcoded or not)
Trying to integrate with Hiero/Nuke, but do all the color tweeking in RCX. The interface is just getting good these days and the conforming capacity is only improving.

But if resolve could have a similar "metagrade/firstlight" window to its color window with controll over all SDK parameter it internally supports, I'd probably stay in resolve all the way...

[Blair S. Paulsen]

This subject can get deep pretty fast so I will just offer one approach that has worked for me on 75% of the material I deal with.

Shoot with the ISO at 800 and the WB at either 3200 or 5600. Move the iris up and down the scale while watching the histogram and look for an exposure that spreads out the values. Use other tools to see if that exposure holds up.

When I open the image in RCX Pro or Scratch I set the WB and tint first, then I try different ISO settings until the mid-tones are just a little low. (Most of the time I end up at 320 to 500 depending on the scene values.) Then I use the curve to emphasize or de-emphasize parts of the tonal range to get the image I desire.

A fairly standard starting point for me is to "stub the toe" at the bottom just enough to find a good black level (never more than a smidgen of crush), move the mid point up until skin tones or other key values seem right, then make a big broad shoulder in the highlights. This creates a curve that is steeper in the 7-8 stops that make up 95% of what can be seen on an actual display in the real world which typically helps separation and contrast, which by extension helps perceived sharpness without adding any sharpening that might bring unwanted artifacts.

Obviously the "secret sauce" is in knowing when and how to deviate from this starting point "formula" to best service the particular image or creative intent at hand.

I further stipulate that this approach is going to throw some information away, but if done well, not much of anything useful. It also allows me to deliver the intended look into the post pipeline in a way that suits the proscribed workflow. If there is going to be a proper final color pass then I keep my curve adjustments more subtle and render out to DPX or OpenEXR in RC3/RLF. If the time and budget for color is limited or nonexistent then I work the curves a bit more aggressively and suggest delivering in a format that can be both offline and hero media - ProResHQ, DNxHD175, etc. For these more "direct to finish" workflows I suggest RC3/RG3 but be careful with RG3; its great on a lot of stuff, but for some images the particular shape of the RG3 gamma curve does not interact well with the specific image at hand, or more critically, applies greater "pressure" on a part of the curve that is weak to begin with. This is not a "red flag" warning, even those images that suffer a little from the RG3 profile will still look pretty close to as good as they ever could have. Plus, most well exposed material looks better with the RG3 gamma curve than RG2 or other options.

Not sure if I have properly followed one of Gunleik's paths of discussion, but I hope I offered a meaningful data point.

Cheers - #19

[Gunleik Groven]

Thanks Blair.

No Cencorship I know of here, and I think I can follow your approach.

I was planning to post some images here with R3d and different development settings and analysis, but if this thread is only read by those who are comfortable with these things... It's not much of a ppoint.

Just planning to take one over and one underexposed Tungsten shot.

Is that helpfull?

At least for people to take a shot at and discuss approaches?

Would in case be a parallell Epic/R1MX shot in 4khd. Could also get into discussing a bit of the differences between them...

Cheers

G

[Ido Karilla]

@ Bruce, as to WB... I Have found that you get fairly undistorted images in the WB realm of 4400 - and 6000. I haven't managed to see any baddies introduced through tint.
So I do agree. WB is a bit of a choice of approaches.

I can def get less noise by staying more centered with WB and ballance it in RGB land.
BUT that is probably not a good GENERAL advice.

Like 800 and WB is a good shooting GENERAL advice, I can see why people choose to shoot 320/daylight.

I will give you a fresh example of miss use of tint.

A production came to me today with Epic footage that seemed very noisy to them.

It was a studio shot of a stage with windows and a green screen behind.

Every thing was lit properly, never the less the stage gray walls where moving around with noise.

I reset the metadata dialed on set and a full green cast was over everything, on set they tweaked the tint so much that the actors looked fine since they where in a uniform light. but the dirty gray walls on the other hand became noise from color oscillations not well maintained by this tweak.

So with a two pass raw grading and some simple composting the shot was saved.

This are the real life situations that small budgets, not so informed crews bring to our desk on daily basis.

[Erich Ocean]

People seem to think of a transcode as a bad thing. It's not.

A properly done transcode is equivalent to caching the debayered output with whatever metadata settings you chose.

Since you rarely, and I mean rarely, change the RAW metadata settings after you've done your one-light, the transcode is effectively a way to speed up everything later by caching the results of those decisions.

Put another way, you never, ever lose anything by doing a transcode -- it's just a cache. If you later decide you chose the wrong development parameters originally, you'd go back to the RAW file from the camera, pick new paramaters, and do a new transcode. (You may have to adjust some creative decisions down the line as a result.) This is the value of shooting "RAW" – nothing more.

If you think of the transcode as a cache, you'll see why it's still a "RAW" workflow all the way down.

There's nothing "cool", "better" or "smart" about repeatedly regenerating that cache because you choose not to transcode your RAW file. You're basically just wasting time and energy (literally). You keep doing a transcode, and then throwing away the results, only to move the playhead over that frame and do it again!

There is no value in not generating the transcode cache up front. None. The only thing you could possibly gain is reduced storage space (by not having to store the transcode cache). But storage is cheap, and processor and your own time are not. The cost of storage is literally the only possible argument against not generating the transcode cache, and it's a seriously weak argument.

(Transcode time is not an argument, because the transcode always happens regardless of whether or not you write out the cached files to disk or not. You can't get away from it.)

[Gunleik Groven]

@ Ido
Yes... I didn't think of that possibility when I came with my tint remark. I generally try to make people shoot as close to native as they can and never tweak the camera like that on-set.
I see how tint can be destructive when used like that.

@ Erich
Kind of one of the main points of this thread.
Thanks for summing it up so consistently. In a multi-editor/VFX-artist/colorist environment, I'd say that a full transcode with proper meta-settings is a very rational and effective AND lossless way of treating the RAW images.
It just isn't rational here to buy in the ballpark of 20 Rockets to serve (allmost) the same purpose....

(allthough we do go without transcodes when that is a rational/practical solution...)

I think it is good to look a bit at the

RAW = magic
RGB = messed up

statements, as both are false in themselves and really not a very interesting way of discussing RED post IMHO...

[Ido Karilla]

For us as a small facility doing mainly feature films, with 2 colorists and one technician the choice of staying RAW is more of an artistic choice.

We feel that having the raw at hand let you experiment with the DP and director more, it is much easier to offer them different approaches to the scene discussed.

I find kelvin a much more sensible and intuitive when following changes in time of day. It can be achieved in RGB but there is something much more subtle when you are defying thing by changing the kelvin by hundreds. It is also a great tool when matching sides that where shot outside at the end of the day with a half an hour between them.

Just my 2 cents.