RAW workflow - a discussion

[Gunleik Groven]

There's a rather heated discussion goimg on somewhere else as to compressed vs uncompressed and what compression really is, as people seem to put a lot of different meaning into the word.

Thoughts?

 

[Adrian Jebef]

This deserves a bump. I'm on the same page as GG.

Back on topic. Here is a respnse to a question in another thread I guess I should post here too, as I guess not everyone are interested in discussing the images I do...

Sensors see light as linear. The response is (matrixed out to be) linear.

Essentially, a sensor is a photon weight. And the data is mapped "linearily" basically meaning that everytime you add a stop of light, that stop is mapped to the "next" bit, which again has twice the theoretical precission as the previos. Kinda different to our eyes who have "a lot" of precission in the shadows, and "not so high" precission in the highlights.

The RAW data, is basically just that, an untreated "sample" of what levels the sensor weighed, and then wavelet compressed in the RED case

Next step is the Matrix, which is a standardised transorm of the data to adjust for nonlinear response in the sensor in the three channels at different intensitylevels in the exposurerange. AFIK this happens on a per channel level. We cannot turn off the Matrix, so we don't really ever have access to the "raw-linear" data. Early on that was an option, but why would we have access to that, other than if we would like to create our own matrix or would like to reverse engineer Graemes magic soup? But you could say that sensor, filtering, compression AD bit-depth and matrix are the differensating and defining elements in a cameras imagecarracteristics. Because it is from there the rgb image-treatment starts.

As far as I can see, the matrix is not "baked" in the raw data from RED, and is worked on behind the scene all the time. But we haven't had any info from RED on that part of the signalchain since the early days. It is visible, though when there are updates in the color sciences. The updates depend heavily on the mateix linearising the images to a "standard" linearlight container.

So. Here we essential have a linear-light-response untreated image that basically looks very "dark" to our eyes, when a LUT is not applied and it is displayed on let's say a rec709 monitor.

Middle grey, which is slightly important to us, actually falls into an area that looks allmost black, which is quite informative as to how and when noise is generated in images. While the highlights (where our eyes precission is not as good) have tons of precissionlevels available, and thus contain less noise.

After this, some kind of LUT is applied, which basically is a RGB transform into a color/luma space compatible with a screening device and our eyes. And different parts of that process is done with different tools.

Which means that tint is adjusted mostly through adjusting saturation values and thus does not affect the noise in the image unless you go to extremes, while white-balance is done through lifting and clamping the red and blue channel, ballancing them around the green. And ISO/exposure changes where the mid-point hits the scale. These transforms can be done in different ways and that is indeed what differensiate different color tools and cameras even more (when the second transform is done in cam) how they implement the different transforms of the linear rgb image and downsample it into something "viewable" (to be a bit simplistic.)

Point is that all the transforms from the original linearlight rgb image, is actually lossy/noise introducing/and destructive, in a "we want it all" kind of mindset.(yup. I know this is not neccesarilly true for "all" transforms, but this is a lineout to get the general picture...) And the only "true" rgb "standard" we have is actually the post-matrix pre-transformed linear light image that any digital camera creates. That is why exr's in and by themselves are containers for linearlight images (yup, i know you can throw "anything" in that container, just trying to be practical here) and the ACES standardisation move, is based on linearlight signals and standard transforms for "any" camera.

Linear is basically the "rawest" signal we will ever touch from a digital camera.
Most manufacturers bake in some of their own soup besides the matrix in that linearlight image too, where iso and whiteballance are the two most common "baked" elements, which I find kinda understandable, because there is little that differensiate a signal as much as those two elements. That is what gives signature, so you can see it "is a Nikon" or a "Canon" or a "Leica" or a "RED" image.

So having some kind of understanding of how these processes affect the images in different systems, is kinda essential for anyone who have to deal with signalchains and workflows.

Anyhow, back to your question.
Yes. It is a setting in RCX, a "gamma option" which basically can give you the unaltered (but matrixed) RGB RAW data directly from the cam, stored in a 16 bit container, so you don't have to worry about downsampling issues.

As far as I have understood on Michael Cionis and others here and in real life, this is the preferef way to start an image chain, if you have the infrastructure and knowledge to treat it well.

But when we say we want to work "RAW", and want "all the data", practically what we are talking about, is this linearlight untransformed 12-bit in a 16-bit container linearlight rgb image.


Now...
I often don't bother... Hahahahaha
I use this knowledge a lot when working on large scale film/tv projects or have ro analyze and solve image-issues for others. Or when prepping with DP's or setting up solutions for grading or VFX for others.
but for me, privately, it is mostly totally overkill to work that way and also not neccesary. I don't have to go "deep" in imagechains to work on my framing and exposure. But your dng question, made me do some tests as to how much noise I could get rid of in heavily underexposed images by starting off with the linearlight signal and not with the gamma-corrected one.

And I was surprised to find the noise so much smoother when corrected from linearlight, than with the RED gammas. Didn't expect that, really, but goes to prove that the signalchains in RCX is constantly worked on, because I have seen different results when doing similar tests earlier on older builds of RCX.

And that is all good.
Lack of noise is not the only charracteristic in a good image. Textures are important, too. And textures are essentially noise. That makes the image come alive.

But that was the long answer to what I was refering to with the last reply...

Cheers

G

 

[Adrian Jebef]

I really only concern myself with the front-end of image capture and processing (mainly on-set dailies) but I need to be versed in multiple back-end (conform/final color) workflow options as I've found the back-end to be the most confusing. There are plenty of conform/final color pipeline techs that really don't understand the process, options, or positives/negatives of their specific workflow choices (Log, Linear, video, raw, etc, etc) for any given project. I get that we all want "best quality" from our original source (R3D, ARRIRAW, etc.) but the pragmatist in me usually finds it's all over-kill. For TV shows or TV commercials creating a pipeline to carry R3D's into OpenEXR into conform/composite into DPX into an HD ProRes master may give you better results but logistically it doesn't line up with turn-arounds and budgets. Conversely, when you do work on the $200 million movie you don't want someone setting up a sloppy R3D to 2K DPX pipeline because they've convinced themselves that if it works for the burger commercial it will work for Die Hard 8. I really think it's project-specific. It's also Post House specific and this explains the grey area in all of these technical discussions which we inevitably end up referring to as "special sauce".

All of these current generation digital cinema cameras work the same way and need to go through the same process at one stage or the other. From a practical point of view shooting raw may be the best (optimal) recording of any digital cinema camera's ability but it also introduces huge complexities that are not easily understood or overcome. There are many on this forum that extol the virtues of shooting raw. There are many who believe that a raw workflow requires more work to understand and that's why others prefer shooting ALEXA to ProRes, and there are those that think they've figured out their raw workflow and insist that their EPIC footage will match or exceed any other camera (including film). Most of the time a workflow choice is branded as "awesome" or "terrible" based on personal preference. Rarely, if ever, are different workflows compared directly against each other. The complexities of a raw workflow are quite a bit bigger than most believe. Fundamentally, the biggest problem with a raw recording is that it's natively unusable. Raw data needs to be transformed/transcoded into a proper RGB image container. The trick is to transform raw data into a large and balanced enough container so you get proper access to the image data. This is when bit-depth becomes really important. When we transform R3D's into 2K DPX 10-bit Log files we are restricting key image data (white balance, bit-depth, top and bottom of the exposure curve, etc). If done properly, these DPX's can be used to produce stunning images but they ARE restricted images. Transforming R3D's into OpenEXR 16-bit linear produces a much larger container to hold some key data ranges. This explains the discrepancies in final grading. 10-bit DPX transforms may have lost subtle color and micro-contrast differences in the image. When a heavy grade is applied artifacts and other color/gamma/sat compromises may become apparent. These deficiencies are usually attributed to the camera/capture system but these same deficiencies may not be present if a larger transform were used (16-bit EXR) instead. Usually I find the differences minuscule. Sorta like the difference between shooting with Ultra Primes vs. Master Primes. But every so often the difference can be HUGE. Especially if you're working with intensive VFX, compositing, grading pipelines.

Personally I believe most projects destined for TV or the Web would be best suited to a Log video workflow. The ALEXA makes great pictures but it's ARRI's Log C ProRes encoding that is the other half of the equation. When I generate dailies in Resolve I can always get excellent results with ALEXA PR Log material even when I apply agressive grades. I attribute most of it to the camera and DP but it's also the simplicity of starting and finishing in a RGB video pipeline. Sometimes working with ARRIRAW material isn't as simple. Visually in Resolve the difference between ALEXA PR Log C and ARRIRAW transformed into Log C is indistinguishable but after agressive grading I can notice some deficiencies in the transform from raw to log video (mainly in some added color saturation or noise). For dailies/editorial it's no big deal but for final conform/color I sense the need for a larger/intermediate transform. This makes the case for ACES and linear-light pipelines when working with natively raw master footage.

And, of course, the last caveat is that it's always easy to place blame on a less-than-ideal looking image but it's harder to figure out why.

 

[Adrian Jebef]

I am kinda tempted/not tempted to dive a final couple of months into pure geekery.
Love the more practical application of image knowledge and to try to learn better and controll better the other sides of image creation besides the purely technical ones...

Hm...

YES! Geek out my friend. I am with you! What do you need? Some still frames from ALEXA and F65 and F55 and SI2K and ? raw files?

 

[PatrickFaith]

Rarely, if ever, are different workflows compared directly against each other. The complexities of a raw workflow are quite a bit bigger than most believe. Fundamentally, the biggest problem with a raw recording is that it's natively unusable. Raw data needs to be transformed/transcoded into a proper RGB image container. The trick is to transform raw data into a large and balanced enough container so you get proper access to the image data. This is when bit-depth becomes really important. When we transform R3D's into 2K DPX 10-bit Log files we are restricting key image data (white balance, bit-depth, top and bottom of the exposure curve, etc). If done properly, these DPX's can be used to produce stunning images but they ARE restricted images. Transforming R3D's into OpenEXR 16-bit linear produces a much larger container to hold some key data ranges. This explains the discrepancies in final grading. 10-bit DPX transforms may have lost subtle color and micro-contrast differences in the image. When a heavy grade is applied artifacts and other color/gamma/sat compromises may become apparent. These deficiencies are usually attributed to the camera/capture system but these same deficiencies may not be present if a larger transform were used (16-bit EXR) instead. Usually I find the differences minuscule. Sorta like the difference between shooting with Ultra Primes vs. Master Primes. But every so often the difference can be HUGE. Especially if you're working with intensive VFX, compositing, grading pipelines.

I just completed a fairly deep analysis of format's and workflow, what the trade matrix unexplectly came down to was what is the primary genre that your studio/production company is focused on. With me, about 90% of all time for a complete production is in pre and post processing, with a 10 to 1 ratio of vfx composite per photograph frame (i.e. 10 frames of vfx for each frame of red footage during compositing). What I found is that heavy vfx workflows, the r3d/linear/openexr gave a better product at a lower cost then any other workflow given my requirements. This would be a totally different if there was a low vfx to "real" footage (i.e. reality tv).

What RED gives me is a super clean way of getting to linear and at a very low cost. Also, though my ratio of vfx to RED production footage is high (10:1 vfx to r3d), in most cases the vfx workflow is started from the linear r3d (for things like camera solves) - so having a clean 4k linear flow that starts the entire pipeline makes the entire process less expensive (i.e. if your starting a vfx pipeline on some compressed junk and the camera solves are icky, the entire production cost hit's a multiplier ... garbage in gives garbage out ... especially when it comes to mathematical functions).

And talking about "complexities of raw workflows" that you mentioned - my studio just finished the workflow move to volumetric lighting and effects ... which was a major effort, but was doable by a small studio because we focused on r3d linear and openexr(vectors and 3d volumes are in openexr, which are often generated by the linear r3d). Again, this probably doesn't make sense for most dramatic features due to their low ratio of vfx to physical footage, but did make sense for us. Another thing, is after one goes through this major effort of a workflow trade study, you will be "stuck" with this workflow for at least a few years. This getting "stuck" on a standard I think is just a business reality the nice thing about being "stuck" on a R3D/Linear & openexr standard, is I am stuck on something that can evolve without breaking my entire pipeline. I.E. a major requirement for the trade is picking a workflow that can evolve without breaking everything. Also you want to be stuck on a standard which is getting cheaper each year (i.e. each year my disks/cpu is getting cheaper for the openexr, and each year the r3d production cost is getting cheaper due to RED's pricing approach).

 

[Lauri Kettunen]

He didn't mention that Raw files only contain 1/3 the amount of information as an equivalent RGB image because they are a reflection of a Bayer sensor (thus one major reason they're smaller), and he also didn't mention that the primary reason for avoiding Raw in later post processes is largely processing overhead associated with both decompression and debayering.

While this makes sense, it is worthwhile to recognize that in post one has far more processing power available than within the camera, and therefore it makes sense to postpone everything "above raw" as late as possible in the chain. The earlier the data becomes fixed, the higher the odds of "good data" becoming unsuable. The more one shoots in nonstandard situations, the more obvious this becomes.