Ask Mike Most Anything

[danibam]

How RAW is it really ?

How RAW is it really ?

I find confusing that the ARRI RAW format (T-Link ARRI RAW) delivers LogC and RED RAW delivers "Linear Light" ...
I keep hearing about quite an amount of digital image processing before a camera delivers RAW data: noise reduction
is one, but with ARRI RAW delivering LogC, I've been wondering if some gamma correction or any other color grading
transformation is actually being done as well? I've heard that ALEXA performs some kind of dual processing of the
RAW capture in 16 bits? Does the ALEXA sensor capture 16 bits? When does it stop being RAW and begins being a
processed image?


Daniel Perez

 

[Evangelos Achillopoulos]

i should not stir up a can of worms but I can't help it. I have a huge problem with people who make numeric claims (as you have) and then, when those numbers are questioned, say "forget numbers, it's all about art." There is problematic logical inconsistency there. Plus, reference projector calibration is first and foremost a technical science. It's not some kind of voodoo. There is a raw and hard technical side to what we all do. It's at the service of art, for sure, but art alone cannot make a good motion picture.

When I'm referring to measurements I'm explicitly talk on what I can control... So in my reference theater, which follows the Color Ttiming critical viewing conditions spec of SMPTE 196M which are:

For laboratory use in color timing, projectors shall have a chromaticity match of x = ± 0.001 and y = ± 0.002. Typical chromaticity readings would be D5500: x = 0.332 and y = 0.347. Chromaticity measurement requires a precision chromaticity meter, not a color temperature meter. Review room screen luminance shall be 55 cd/m2 ± 7 cd/m2 (16 fL ± 2 fL) at the screen center. The luminance of the screen sides and corners, measured as described in 5.2, shall be at least 80% of the screen center reading.

And in these conditions I have the results that I'm stating... +/-1 printer light...

In real world projections though the spec is NOT so critical... to a point that is almost subjective the result... SMPTE 196 spec says...

For theater nominal 35- and 70-mm prints, the light reflected from the screen in theaters shall have a spectral distribution approximating that of a blackbody at a color temperature of 5400 K + 600 K -- 200 K, the use of short-arc xenon light sources being assumed. Theater screen luminance at the screen center shall be between 41 cd/m2 (12 fL) and 75 cd/m2 (22 fL). Luminance at the screen sides shall be 75% to 90% of the screen center luminance, but not less than 34cd/m2 (10 fL).

FYI, 800K error range is about 5-6 printer lights...

Also read what David and Mike say about the subject of relativity in end user viewing devices... in the same thread in page 3....

There's nothing you can do about people watching stuff on badly set-up displays. Nothing. But that doesn't excuse one from working towards a standard -- you don't want a to finish a project at a post house where it only looks correct at that post house, any more than you want to create a movie print that only looks correct at the lab that printed it. At least if there is a standard, there is the hope that display devices will move closer to that standard.

There have always been periods in display technology where it's the wild west, then standards emerge, and then later they are abandoned when a major change in technology happens, then new standards develop, etc. More and more people watch stuff on computer monitors and I think there is a movement towards more common display standards being driven by the content providers.

But ultimately you have to ignore the fact that some people are going to watch things improperly displayed or transmitted, you can't really take that into account unless there is a common and consistent form of misrepresentation. If that happens, then perhaps you can compensate for that -- for example, like in the 1960's when prints were adjusted for dim drive-in movie theater projection, and even lighting styles were affected by that trend.

You can only control what you can control. Nobody can go into everyone's home and calibrate their displays, and that's always been the case. In fact, I find that modern flat screens are far more consistent than CRT's ever were, even with the automatic corrections that so many consumers seem to just leave on all the time. There are standards, even today, and that's what one has to use as a reference, even if it's only used by those of us who have to create this stuff. For better or worse, material that is targeted to video type distribution is corrected and QC'd on monitoring that is set up to Rec709 accepted practices for HD images, a standard that is basically based on CRT's but has been carried over to modern displays with some consistency. That's what gets delivered. What happens after that is not controllable by any of us, and one just has to accept that.

Incidentally, this is much worse within the industry than it is in the consumer world. You've got colorists working on proper calibrated monitoring in proper environments, but you've got DP's and directors looking at dailies on everything from laptop computers in bright sunlight, to iPads, to video village monitors, to tiny CRT's in the camera truck, to home TV's that are set up to who-knows-what. And you've got studio execs looking at digital dailies that have been compressed beyond recognition and are playing in postage stamp sized windows on a computer screen and blown up to full screen size. The whole thing would be funny if it wasn't so ridiculous.

So please realize what you say and what I'm trying to say... "there is nothing absolute in life all are relative and art is the most relative of all..." if this is not good for you... I can't do anything more...

 

[Rob Ruffo]

It sounds like you're referring very specifically to behavior in RedcineX. That represents only one set of circumstances. Many other programs can be and are used to directly access R3D files, and many do not behave in the way you've described. In my view, increasing saturation in a post grading environment is usually not a good idea as it increases noise, particularly in red tones. Many, if not most, colorists would much rather decrease saturation than increase it for that reason. There is some debate on this issue, but in either case, you cannot use the characteristics of one program - particularly one that is working on the debayer parameters rather than post processing - as a basis for general assumptions about image processing, regardless of the camera.

Agreed, I guess I should have specified: I mean saturation settings in direct-raw adjustments made in direct-raw software - I have (I think I have) seen this phenomenon in Resolve, PPro, and Redcine X when looking at RedRaw directly.

 

[David Mullen ASC]

I find confusing that the ARRI RAW format (T-Link ARRI RAW) delivers LogC and RED RAW delivers "Linear Light" ...
I keep hearing about quite an amount of digital image processing before a camera delivers RAW data: noise reduction
is one, but with ARRI RAW delivering LogC, I've been wondering if some gamma correction or any other color grading
transformation is actually being done as well? I've heard that ALEXA performs some kind of dual processing of the
RAW capture in 16 bits? Does the ALEXA sensor capture 16 bits? When does it stop being RAW and begins being a
processed image?


Daniel Perez

I could be wrong but I believe the Alexa sensor has two 14-bit A/D processors per photosite at two different gain levels to create a wider dynamic range image that is merged into a single 16-bit file that is then converted to a 12-bit ARRIRAW file -- as for the Log-C business, I thought that RAW by definition didn't have gamma curves added, but some online info suggests that ARRIRAW does in fact store 12-bit data as Log-C, which holds the same dynamic range info as the 16-bit linear RGB file that was used to create the 12-bit Log-C ARRIRAW file.

 

[Rob Ruffo]

How often do you use sharpening on Red or 35mm footage? Do you think this is a frequent or advisable practice?

 

[Marc Wielage]

How often do you use sharpening on Red or 35mm footage? Do you think this is a frequent or advisable practice?

I have used scene-by-scene sharpening on about 20 35mm D.I. projects over the last six or seven years, but there isn't a "one-size-fits-all" approach that works for everything. In the case of one feature, the DP correctly said (after viewing tests), "sharpness is not our friend," and we used lots of defocus on many close-ups... the opposite of sharpening. Grain management was used on every single project just to make the grain consistent. All the 2K D.I.'s went through a 2K -> 4K uprezzing which was essentially a final line-doubling pass going back out to film, but it's not traditional enhancement or aperture correction per se.

I know of several major digital projects where careful sharpening kernels were used at various points in post, but I can't reveal them (some are currently in progress). One that got some publicity was Avatar. All 3D features need some image processing just to realign and correct for parallax error, interocular distance issues, and general alignment problems, but sometimes sharpening and defocus is necessary just to keep the eyes focused on specific foreground objects. Many, many times we've had to make adjustments to further reduce depth of field in problem shots.

I would be extremely careful about sharpening without lots and lots of workflow tests. Bear in mind that sharpening can encompass H-only, V-only, H&V, plus various combinations of bandwidth settings (highlights only, mids-only, etc.). Diagonal detail enhancement can result in aliasing issues, as can certain kinds of movement, so sharpening is a really, really tricky area that's fraught with problems. I've seen more than one project kicked back from home video QC because of over-enhancement -- sometimes done long before we got the pictures for color-correction, so nothing could be done to fix it.

I think if any enhancement is done, you should do it at the final stage, and decide how far to go based on what it looks like on a 100% reliable monitor or projector. In almost every case, the general rule of thumb is: less is more.

 

[danibam]

I could be wrong but I believe the Alexa sensor has two 14-bit A/D processors per photosite at two different gain levels to create a wider dynamic range image that is merged into a single 16-bit file that is then converted to a 12-bit ARRIRAW file -- as for the Log-C business, I thought that RAW by definition didn't have gamma curves added, but some online info suggests that ARRIRAW does in fact store 12-bit data as Log-C, which holds the same dynamic range info as the 16-bit linear RGB file that was used to create the 12-bit Log-C ARRIRAW file.

But ARRIRAW delivers a Bayer Pattern, right? ... I think I can understand how the output of two A/D processors per photosite could be later merged without first deBayering to RGB, but I am curious to find out if a LogC gamma adjustment can be properly achieved while still in Bayer Pattern ... if it requires any special treatment in order to achieve the same result of a post deBayer RGB gamma adjustment ... and in general if any other color matrix operations can be "safely" performed on a Bayer Pattern.


Daniel Perez

 

[David Mullen ASC]

But ARRIRAW delivers a Bayer Pattern, right? ... I think I can understand how the output of two A/D processors per photosite could be later merged without first deBayering to RGB, but I am curious to find out if a LogC gamma adjustment can be properly achieved while still in Bayer Pattern ... if it requires any special treatment in order to achieve the same result of a post deBayer RGB gamma adjustment ... and in general if any other color matrix operations can be "safely" performed on a Bayer Pattern.


Daniel Perez

I found out today from ARRI that 12-bit ARRIRAW is not Log-C, it is still RAW (Bayer mosaic).

 

[M Most]

But ARRIRAW delivers a Bayer Pattern, right? ... I think I can understand how the output of two A/D processors per photosite could be later merged without first deBayering to RGB, but I am curious to find out if a LogC gamma adjustment can be properly achieved while still in Bayer Pattern ... if it requires any special treatment in order to achieve the same result of a post deBayer RGB gamma adjustment ... and in general if any other color matrix operations can be "safely" performed on a Bayer Pattern.

I think it might be instructive to go through the differences between RAW and debayered images, and also the nature of what "log C" is. RAW data is just that - it is the record of the values of each photosite on the sensor. Now, how that's handled through a compression process, a reconstruction process, and a recording process are different depending on the manufacturer. Until the RAW information is reconstructed as a full RGB image, it exists only as pure data. But data compression has efficiency limitations unless you optimize the data for that particular compression. So in the case of Red, for instance, some transformations are likely made to the RAW data in order to enable more efficient wavelet compression. There are a lot of things you can do in this regard, and techniques include applying a mathematical log curve, computing color channel differences to reduce the amount of pure data needed, and other transforms designed to allow a compression engine to work more effectively. It is still the basic RAW data, but it isn't necessarily the exact values that came off the sensor in their original form. There's nothing wrong with such an approach, it is an effective use of the tools involved. In the case of Arri, similar transforms might be used for Arri RAW, but that doesn't mean they have anything to do with what you would call LogC, or final image processing in any way. What is being done is no different than what has been done using "pre-emphasis" in video cameras for years. That was done primarily by taking higher frequencies - which would often "get lost" in normal processing - and raising their value using a specific known transform so that when the signal is passed through normal processing the values are retained and thus the signal to noise ratio is increased. In modern digital cameras, the problem is not really signal to noise, but storage and compression efficiency, so techniques are used to make that a smoother, more effective path.

Now, when image reconstruction is done, the considerations are different. At that point, what's needed is the most accurate interpretation of the original image that was captured. And further, a way to pass the information as completely as possible to image processing pipelines. In that world, real time processing is often needed and thus the data rate must be limited to a practical size. Traditionally, 10 bit image containers have been used, but with the increase in processing power developed over the last few years, it is probably practical to increase that today, hence the talk about 12 bit and 16 bit containers, as well as the half float container proposed by the current IIF/ACES specification. But in any of these formats, to efficiently fit a large amount of image data into a limited sized container, log coding is very effective, because it is similar to the human response system, and therefore allows for "pre emphasis" of important image detail regions and less precision to describe extremes that the eye is not as sensitive to. Log C is in effect another incarnation of the Cineon log curve first developed by Kodak for digitizing film negative. It is a 10 bit log coded container that effectively can store 14 bits of linear image information. RedlogFilm essentially uses an identical curve, which is one reason you can use the same LUT for either one and get good results. However, because Red has opted to perform the image reconstruction outside of the camera, the image supplied when using RedlogFilm already has a color matrix applied, which is done during the image reconstruction. In Arri's case, when video output is used - as it is for either direct video recording or on board ProRes recording - the reconstruction processing is done inside the camera, so what is provided is a fully reconstructed RGB image. If LogC encoding is used on that image, it does not have a color matrix applied unless specifically chosen, which is why the LUTs you create using their Web based LUT builder can either incorporate a matrix or not. But if you're recording Arri RAW, you're creating much the same situation as with Red, except for the compression part. So at that point, you don't have an RGB image, but you have information that might - and probably does - differ from the exact values coming off the sensor because of transforms used to increase storage efficiency, NOT actual image processing.

Hope that makes sense. I got a bit long winded there, but I've been known to do that...... :001_unsure:

 

[M Most]

Agreed, I guess I should have specified: I mean saturation settings in direct-raw adjustments made in direct-raw software - I have (I think I have) seen this phenomenon in Resolve, PPro, and Redcine X when looking at RedRaw directly.

That's very likely, because all of those programs incorporate Red's SDK, which performs those functions identically to Redcine X prior to handing off the resulting RGB image to their own color processing engines. Red has done that by design, so that any software you use will yield the same results and use the same reconstruction algorithms, which in turn produces consistency across software platforms, as well as a simple color management system.

 

[M Most]

How often do you use sharpening on Red or 35mm footage? Do you think this is a frequent or advisable practice?

I tend to stay away from sharpening in post unless specifically requested or absolutely required. It tends to produce a lot of artifacting if not handled in very small doses. In the film world, some sharpening is usually done in the scanner, particularly for 2K (and lower) scans. But in a grading environment, I've usually found that nearly all sharpening algorithms yield very "edgy" results, much like cheap video cameras. You can, of course, restrict it to specific areas of the image, such as the eyes, for instance. But while I agree that sometimes it's useful, for most properly shot images it often does more harm than good. Only my opinion, though.

 

[Marc Wielage]

But while I agree that sometimes it's useful, for most properly shot images it often does more harm than good. Only my opinion, though.

I would also add that I have seen many, many projects get kicked back from QC because of over-enhancement. It's way too easy for people to get carried away with excess sharpening. This can be a disaster for small features trying to deliver final masters to fulfill a distribution contract.

In my case, I've generally not had to use it more than (say) a dozen shots in a feature, but on those specific shots, it helped quite a bit. There are proprietary sharpening processes used by a couple of restoration companies I know of that don't apply enhancement in the usual way. When it's preceded by grain reduction, you can actually get away with more sharpening than you might think. The Bond movies on Blu-ray and the recent Star Wars video releases are good examples.

But this is not the same kind of tool as what's available in DaVinci, Baselight, or any conventional color-correction system. Enhancement can be a very, very dangerous tool in the wrong hands -- not unlike noise reduction. Too many people think if a little is good, then a lot is even better. Not true with enhancement and NR.

 

[Nathan Rosenquist]

Hi Mike,

Thank you for all of your contributions to this thread. We are lucky to have someone with your experience sharing their knowledge so freely.

I have a question about replicating the look of optical diffusion filters through digital means.

The conventional wisdom, as I understand it, seems to be that if you want to approximate the look of various optical diffusion filters you can get fairly close in post by copying your footage onto a new layer, applying something like a gaussian blur, and then lowering the opacity of the blurred image. In this way, you have both the sharp image underneath, and some of the blurred image blended in to smooth things out. Then both the blur and the opacity of the blurred level can be adjusted to taste to fine tune the effect.

Whenever I have tried this, it seems to work pretty well if all you want to do is soften the image up a little bit. However, I have found it much more difficult to replicate any of the other optical artifacts that some filters have (ProMist, etc.) In particular, it would be nice to be able to have selective control over halation in light sources. With the technique I described, there isn't really much halation at all. I also recall David Mullen saying in another thread that he shot a film (Big Sur) where they had only the blacks halate, drawing inspiration from older film techniques.

I'm especially interested in the general principles behind this, as opposed to an all in one plug-in or system that has a preset, as I'm trying to understand the differences at a more fundamental level.

Do you have any tips on how to gain better control over this process?

Thank you for your time!

 

[M Most]

I have a question about replicating the look of optical diffusion filters through digital means.

The conventional wisdom, as I understand it, seems to be that if you want to approximate the look of various optical diffusion filters you can get fairly close in post by copying your footage onto a new layer, applying something like a gaussian blur, and then lowering the opacity of the blurred image. In this way, you have both the sharp image underneath, and some of the blurred image blended in to smooth things out. Then both the blur and the opacity of the blurred level can be adjusted to taste to fine tune the effect.

Whenever I have tried this, it seems to work pretty well if all you want to do is soften the image up a little bit. However, I have found it much more difficult to replicate any of the other optical artifacts that some filters have (ProMist, etc.) In particular, it would be nice to be able to have selective control over halation in light sources. With the technique I described, there isn't really much halation at all. I also recall David Mullen saying in another thread that he shot a film (Big Sur) where they had only the blacks halate, drawing inspiration from older film techniques.

I'm especially interested in the general principles behind this, as opposed to an all in one plug-in or system that has a preset, as I'm trying to understand the differences at a more fundamental level.

Do you have any tips on how to gain better control over this process?

You can achieve specific effects by qualifying the area being affected by the 2nd layer. Luminance keys are the usual approach. By keying only the bright areas, halation effects can be had by varying the amount of the blur on the additional layer, and also by using different blending modes (add, screen, etc.). Conversely, the effect David was after can be achieved by inverting the key and affecting only the darker areas. These things can get considerably more involved, but basically that is usually the approach. Any one-button solution is going to have limitations. Doing it "the hard way" gives you a lot more control, so it's usually how professional colorists do it, both because of the superior results and because we're basically masochists.

 

[Marc Wielage]

The conventional wisdom, as I understand it, seems to be that if you want to approximate the look of various optical diffusion filters you can get fairly close in post by copying your footage onto a new layer, applying something like a gaussian blur, and then lowering the opacity of the blurred image. In this way, you have both the sharp image underneath, and some of the blurred image blended in to smooth things out. Then both the blur and the opacity of the blurred level can be adjusted to taste to fine tune the effect.

It's an ongoing debate as to whether to use filters during the shoot, or do it all in post. That goes for color, diffusion, grads, whatever. DPs often make a good point that doing it optically during photography is a more "organic" method, plus it bakes the look in to the point where nobody can take it out later.

My problem with that coming from post is that if they go too far with the filters in production, it's a huge problem to try to reduce it (or worse, take it out, if somebody change their minds). You have infinite control in post over how intense or subtle the effect is -- defocus, sharpening, tint, desaturating, etc. -- but that also means the DP has the potential of having to surrender control to somebody else later on.

Note also that using filters during photography can create problems with blue screen, 3D, and other VFX situations. The best answer I think is just to do a workflow test, try it both ways, and see what works best for you. If it were up to me, I'd almost always say, do it in post -- provided you have a decent budget and schedule for color-correction.

Mike above is absolutely right that pulling defocus keys has become standard procedure for a lot of stuff these days, especially with actors "of a certain age" and also controlling things like blown-out highlights and so on. A filter would treat everything more or less the same way; a defocus key could attack whites separately from everything else, skin tones separately from everything else, plus you could create garbage mattes (windows) to contain the effect only where you need it.

 

[M Most]

Note also that using filters during photography can create problems with blue screen, 3D, and other VFX situations. The best answer I think is just to do a workflow test, try it both ways, and see what works best for you. If it were up to me, I'd almost always say, do it in post -- provided you have a decent budget and schedule for color-correction.

Opinions are always based on one's own specific experiences, so perhaps surprisingly or perhaps not, I would probably say exactly the opposite, unless visual effects are involved - and yes, I consider stereoscopy a visual effect. But I'm basing that on the fact that I have had the good fortune of working with a lot of very experienced, very talented cameramen, most of whom started their careers before these kind of post techniques were commonplace, and thus have a very good understanding of what they're trying to achieve and how to achieve it. It's not necessarily the right answer for everybody.

 

[David Mullen ASC]

I've had less than satisfactory results trying to do digital diffusion for an entire show (as opposed to the occasional shot) -- I find it time-consuming and hard to keep the sharpness while getting that hint of halation around sources that I want. Whereas I know that if I use a 1/8 Black Frost, for example, that it won't impact sharpness much. With blur overlay techniques, it's hard to increase the halation while not blurring highlight details. I know it's possible but it so much simpler to just use the camera filter.

But my general principal is to under-do the level of optical diffusion and not use it at all on extreme long shots, or telephoto shots, or efx shots, etc. knowing that I can always add more diffusion in post. But for your run-of-the-mill dramatic scene where a mildly filtered look is desired, I don't see a problem with doing it in-camera as long as you don't over-do it. And the truth is that occasionally you get happy accidents with optical diffusion which is one of the reasons you use them, that sort of optical interactivity inside the glass, which feels very organic, just like how I prefer real lens flares to digitally-added ones.

 

[Michael Lindsay]

Hi Mike

How, witin the traditional telecine controls offered in systems like mystica/speedgrade (specifaclly those that do not have curves), is one suposed get fast and steep toe and shoulder compression that allows one to produce a really contrasty picture that keeps as much shadow and highligt details as possible?

In days gone bye I would ask for a particalur setup of the the leagliser (on the back of the TC suite) to take some of the super whites and blacks back in to the pictuture (heavily compressed). It was never particaully satisfactory but it would work a bit.

I am in a colour session on Mystica(s3d job) on monday and I am guessing I might have to resort to a fudged use of imported LUTs to provide what I want?

thanks for any thoughts you offer

Michael

PS I believe we once met in Miami. I was getting a tour of a facilty that was doing some processing for me and you where sitting at a scratch seat that was just newly put in...

 

[David Mullen ASC]

I'd also add that if you were shooting a TV series, with a ton of footage being generated, and the mandate was to make the female stars look as good as possible, saving any diffusion for the end of post wouldn't fly if you are going to be judged mainly by how it looks on the HD monitors on set and how it looks in dailies. By the time post is finished and the series goes to air, the DP may be long gone if he didn't make the female leads look good.

 

[M Most]

Hi Mike

How, witin the traditional telecine controls offered in systems like mystica/speedgrade (specifaclly those that do not have curves), is one suposed get fast and steep toe and shoulder compression that allows one to produce a really contrasty picture that keeps as much shadow and highligt details as possible?

In days gone bye I would ask for a particalur setup of the the leagliser (on the back of the TC suite) to take some of the super whites and blacks back in to the pictuture (heavily compressed). It was never particaully satisfactory but it would work a bit.

I am in a colour session on Mystica(s3d job) on monday and I am guessing I might have to resort to a fudged use of imported LUTs to provide what I want?

I don't have any real experience with Mistika, so I can't offer any specific suggestions on that. Some programs - Scratch comes to mind - have S-curve controls that offer a continuously variable S-curve, which would to a great degree accomplish what you're looking for. A lot of what you're talking about really comes down to having experience and touch. I've mentioned him before, but Stefan Sonnenfeld creates almost all of the looks he's famous for using not much more than lift, gain, and a few cleverly placed windows. It's not always about having specialty controls. It's often about using the basics to their utmost.