Black Shading:A look under the hood

[Gavin Greenwalt]

I have a much better proposal but maybe it's technically impossible, why can't we store blackshades and load them into REDCine directly? Is it just because of compression? Then you could match up by Serial number and run a dozen at different framerates and let REDCine apply it to the RAW footage.

 

[David Battistella]

The blackshading affects how pixels see and absorb light so I think it has to be done in camera because it affects capture.

Generally you can move the shutter about two spaces in either direction with whatever blackshading in in place. So a 172.8 degree shutter calibration gives you 5 shutters to choose from. I hope thins thread doesn't turn into blackshading panic, that wasn't the idea, it's more to show the sensors normal state after four different kinds of blackshading calibrations. For example looking at 360 at 24fps and 172.8 at 24 FPS, I wouldn't feel so compelled to do a blackshading between those frame rates now that I see the pattern from both. It's all about figuring out when and why I would absolutely do a blackshading calibration.

Obviously between 1 FPS and 24 FPS, you are going to have to do it.

So it's about planning how to use the now ample four slots for blackshading in preparation for conditions going into a shoot. Later today I'll be mixing 24, 48 and 96 FPS so I'll plan that out accordingly and have them mapped.

I mostly want want to know what frame rate changes will affect black calibration the most and look drastically different so I can be ready. On MX I knew when that was going to happen, I'm figuring out on dragon and wanting to share my results.

Battistella

 

[David Battistella]

Fair enough.

The reality is black shading will give "optimal" results. Many people ride with one black shading and don't make a lot of switches to framerates over the course of a normal shooting day. I think REDs actually telling how to get the very best out of the camera, which I appreciate, so I work out how I personally want to manage that. It might be complete overkill on my part. (And it probably is) but I don't want to loose shots because I missed something.

Since blackshading is specific to each sensor I don't think there is a blanket approach so red provided us with more slots to save us time when we've planned things out correctly.

Battistella

so I don't see this as a problem but a way to max out the quality of the pixels I capture and to better understand how to approach that on dragon.

Ps. I think your comments are constructive and I wasn't pointing any fingers. The whole black shade thing tends to cause panic when it's not that big a deal. Just about every camera has a black balance, just reds takes a bit if time.

 

[Mark Phelan]

The calibration maps are tuned per sensor, you cannot apply from one sensor to another. We actually block this in the FW.

I love learning this sort of thing here on Reduser. I suppose I always thought as much, but knowing it helps to better understand the technology. It also tells me the cameras do have personalities a bit, which is not surprising.

 

[luigivaltulini]

Blackshading do them only after a firmware change, since I had Dragon or Epic.
I just had to do outdoors in the cold-10c or timelaps for a single night.
My cameras are often out to hire anyone, and never came back with problems of shading in blacks.

 

[Mark L. Pederson]

I have a much better proposal but maybe it's technically impossible, why can't we store blackshades and load them into REDCine directly? Is it just because of compression? Then you could match up by Serial number and run a dozen at different framerates and let REDCine apply it to the RAW footage.

I'm pretty sure that the CalMap is applied to image processing BEFORE compression + recording - which would make the most since as you don't really want to compress un-needed noise.

 

[D Anderson]

This is all very interesting. It does sound like RCXP is not the place to apply BS unfortunately. 4 stored BS on camera is a good start.

 

[David Kuhn]

The blackshading affects how pixels see and absorb light so I think it has to be done in camera because it affects capture.

Unless RED is doing something exotic (possible), blackshading is the business term for offset correction or DSNU (Dark Signal Nonuniformity). Offset correction doesn't actually alter how the pixel captures light, rather it's a subtraction applied to each pixel right after the frame is captured. Generally, a camera frame is captured with no light and the value of each pixel is captured and stored (typically in an array called a Look Up Table... LUT). That LUT is then stored on the camera and subtracted from each subsequent frame... after which each pixel would then read zero when there is no light.

It's one of two image corrections that are typically used to correct for pixel to pixel variation (fixed pattern noise)... the other is Gain correction (also called Photo Response Nonuniformity by academics - had to look that up) which doesn't seem to be used much outside of machine vision (or at least it's not customer applied... may be done internally). Quality still and motion cameras don't seem to need it, certainly not to the extent that machine vision cameras typically do (they often exhibit easily visible pixel to pixel variation looking at a white reference standard). When I perform gain correction I image a reference white (a 'Calibration Standard' with known spectral response illuminated with the same light source I'm going to use after calibration) and capture that image (usually a lot of images that I average to reduce measurement error), the resulting pixel values are then divided by a target white level (some distance below saturation... I often go with 3800 or so on 12-bit images). That LUT is a normalized gain table and many machine vision cameras have the ability to multiply values in the gain lut by each pixel... resulting in a uniform output from each pixel in a sensor.

 

[Meryem Ersoz]

With all of these available frame rates and resolutions, I like this idea of storing a broad range calibrations on a piece of media, then summon them up quickly.

My question is, is it possible to store looks and black shades on the same piece of media? i usually have kept my looks stored in redcine, then offloaded them from the software, as needed. but if i could put it all in one place, that might be the way to go.

also would love it if RED could produce a smaller, cheaper piece of media which we could purchase for this purpose. this is currently a rather pricey use of media and certainly doesn't use much storage space. I know this has been brought up in the past, but as black shading maps become more needed, something cost-effective might encourage people to use the function more effectively, resulting in cleaner representation of RED footage.....

 

[luigivaltulini]

Sweet Jesus is it best to say nothing at all?

The problem is the need for BS.

So luigi, you don't shoot slow-mo or high speed shutter....ever?

If you do then you just ignore the temp warnings? Or you have a magic camera that does not need a BS?

always change the frame rate from 25 fps to 120 ...
As far as I can see the important 'that the camera uploads the temperature.
I have much difference when they are cold in the snow.
I always use adaptive mode.
However Ivan, I think that Toia not make blackshade from what I remember.
So are not the only


My 2 cents

 

[Chi Ying Chan]

Actually I always did!!!
Don't see any differences !!!

Regards
Chan Chi Ying

 

[Chi Ying Chan]

From 1 f/s to 300 f/s , no differences, no report from onset DIT, no report from Post house.
From Red One to Epic, I BS each day before shooting.

Regards
Chan Chi Ying

 

[Chi Ying Chan]

We r shooting real 3D with two rigs.We keep changing speed and shutter angle due to action scene,besides DIT, we has a QC department monitoring the footages,the most common report from them r "can u fill a little bit more, good for post" :-(

Regards
Chan Chi Ying

 

[Björn Benckert]

that doesn't surprise me at all, and im surprised you're surprised frankly



the photosites on the sensor are all of different sensitivities (a product of manufacturing etc), so the blackshading determines by how much they vary, so they can be "evened out" before being saved.. different exposure times produce different variations

1/sec exposures often generate hot pixels (full intensity , normally pure R, G or B)


..but this has nothing to do with the difference you see between the noise levels you see with different exposures ..thats purely down to exposure time..

longer exposure times collect more light = less noise

thats why you light dark scenes.. a stop is a stop.. doubling the exposure time..makes a stop of extra light available (to the sensor) =less noise)

a 1/ sec exposure can gather 6+ stops of extra light..50x as much as 1/50 (ie, you shot 360deg @ 1fps = 1sec exposure)

also..the preset blackhadings are a lifesaver, but never going to be as good as a fresh calibration (environment, internal and external heat..all change the results).. but its better than not doing one

swapping maps is NOT A GOOD IDEA!!
.. blackshadings are relevant for individual sensors only..thats why each camera can do their own..they need to


ps, do people really still shoot 24fps ..wow .. id just shoot 25 for international release (slow audio by 6%)..or 25 or 30 for domestic.. you romantic fools

You are the fool.... Try shooting 25p in the states where the electricity is 60hz... When the flicker hits you, then you problably want to change this statment... also 24p is far better for international release than 25p and it also a very valid point to say that 24p gives you a better picture as your clmpression is lower per frame. Also when you broadcast it needs 1/25 less bandwidth or you get 1/25 better looking picture...

Also film picked 24p for one reason, it looked good. 30p,25p 50i and 60i abd all the other framerates where invented to fit the electric power standard...

 

[D Anderson]

Also film picked 24p for one reason, it looked good. 30p,25p 50i and 60i abd all the other framerates where invented to fit the electric power standard...

I didn't know that for some reason!

 

[Mark Wilkinson]

Is there some new info about needing to BS every time we change frame rate or shutter angle?

The current DSMC guide doesn't say so:

1. Calibration is required:
   
   •  After an extreme change in temperature (+/- 30°F or +/-15°C) from the currently selected calibration map.
     
   •  After an extreme change in exposure time (+/- 1/2 sec) from the currently selected calibration map.
     
     

While I admit that theoretically it might make some difference - where did we get the idea that we had to do it for shutter angle or frame rate?

Please advise...

 

[D Fuller]

Theoretically, there are only two variables that matter: SENSOR TEMP (not the outside temp, that has nothing to do with this) and EXPOSURE TIME (not frame rate).

But there is something odd about David B's results at 1fps. If there is a noise difference between 360 degree shutter and 172 degrees, it suggests that there is another variable at play, like down time for the sensor, perhaps. And it may be a red herring in this discussion of black shading. (Meaning it may have nothing to do with black shading, but may affect noise on its own.) Is there something about giving the sensor or the other circuitry a rest for half a second that has a quieting effect on noise? Letting residual charge dissipate? I have no idea, but it seems equally possible that the difference is unrelated to black shade.

 

[Mark Wilkinson]

I think keeping shutter angle relative and going to 100fps would count as "an extreme change in exposure time"

Nowhere near the stated +/- 1/2 second...

This may be a rabbit hole created by Dragon's ability to save cal maps making us suddenly think that's what we should be doing.

 

[Mark Wilkinson]

Ah. Thanks.

 

[Gavin Greenwalt]

I'm pretty sure that the CalMap is applied to image processing BEFORE compression + recording - which would make the most since as you don't really want to compress un-needed noise.

Makes sense for heavily compressed footage but in many situations with ~5:1 compression or less I would think a "safe" black shading that kills fireflies and black pixels would be sufficient to keep the compression reasonable and then fine tune with a set of 'offset' cal maps.

So do like a 1/4s shutter blackshade in camera, and then fine tune from there with offset cal-maps to polish off the exposure.

Or similarly, it seems like you could record 4 cal-maps and blend between them based on the shutter. So roll off a 1/96th, a 1/48th a 1/24th and a 1/6th and blend.