What’s the color resolution and diagonal resolution of the RED ONE?

[Chris Davis]

Hello,

I’m a video hobbyist trying to better understand how a Bayer sensor like on the RED ONE works. I posted here rather than in the RED ONE section because it appears that threads in that section are more about how to use the RED ONE, and this section has more threads about the technology regarding how these cameras work. I'm interested in buying the Scarlet, but the RED ONE is the camera that people have actually tested.

I read an article and watched some videos by John Gait, who helped develop the Panavision Genesis.
http://magazine.http://www.teenink....le/the-truth-about-2k-4k-the-future-of-pixels
http://www.panavision.com.au/News/Demystifying_Digital_Cameras.htm

It got me thinking…

I don’t understand how a Bayer sensor, four thousand pixels wide, can produce 4K of color resolution. When I look at the sensor, it basically looks like it’s made up of blocks of four pixels – one red, two green, and one blue. I understand that by interpellation a chip/computer can calculate what color might been at that pixel from looking at the surrounding pixels. But it looks like the actual color resolution would be something like 2K. What am I missing?

Also, Wik says that on a Bayer sensor the green doubles as a “luminance-sensitive” whereas the red and green are “chroma-sensitive.” So I can see how the sensor would provide full horizontal/vertical luminance resolution. If the sensor is turned diagonally at a 45 degree angle, however, the green photosites are every other vertical “line.” It looks like something like this would provide about twice the horizontal/vertical luminance resolution than it would diagonal luminance resolution. Is there anything to this?

I don’t mean in any way to imply that the RED ONE is anything but an incredible camera. From everything I’ve read, it definitely is. I’m just a bit curious about he nuts and bolts of this.

Thanks,
Chris

 

[Graeme Nattress]

Well, perhaps take a look at this: http://www.cinematography.net/natress-red-res.html

and: http://www.cinematography.net/RED One Resolution/A012_C003_120258_001_crop.png where you can see a zone plate pixel for pixel.

You can see how the luma resolution works, and how the R, G and B pixels work together to create a full resolution image. For a pure red or blue colour, resolution is limited to 2k, but appears much higher anywhere the colour is less than totally pure.

In the end, you're always getting at least 2k resolution on R and B, and you can see detail "just" out all the way to 4k in luma / green, but we call it at around 3.2k, which is obviously significantly higher than any video camera.

I have no problems talking about how Bayer pattern colour filter arrays work, and to show you the images such a scheme is capable of. I'm also happy that if you want a perceptually high resolution / nice looking image, it's a more suitable pattern colour filter array to use than the RGB stripe colour filter array that Genesis uses.

Graeme

 

[David Rasberry]

One might add that a Bayer pattern sensor mimics the way the eye naturally resolves luminance and color too. I.E the eye is more sensitive to green and less sensitive to red and blue. The luminance to chrominance ratio is very similar.

 

[Chris Davis]

Graeme and David,
Thanks. Graeme -- I'll look over the links this evening and get back with any further questions.
-Chris

 

[Chris Davis]

You can see how the luma resolution works, and how the R, G and B pixels work together to create a full resolution image. For a pure red or blue colour, resolution is limited to 2k, but appears much higher anywhere the colour is less than totally pure...
I have no problems talking about how Bayer pattern colour filter arrays work, and to show you the images such a scheme is capable of...

Thanks for the link, but I’m having a difficult time understanding how the luma res works and how the RGB pixels work together from this page. Maybe the info’s on a different page on this site? If you know of any other links that might help me really understand the way interpolation of a Bayer like on the RED ONE/Scarlet 2/3” sensor works, I would love to read them.

Also, I am curious about diagonal resolution. When I look at a Bayer pattern, it appears that the green/luma pixels are checkerboarded between two horizontal lines, so that, if you look at two horizontal lines, there’s a green pixel on one or the other of the horizontal lines all the way across. So I can see how it would get the high horizontal/vertical resolution. But if the sensor were turned 45 degrees, the green would be every other line. Does a Bayer sensor get less diagonal luma resolution than horizontal/vertical?

Thanks for your time,
Chris

 

[Dan Hudgins]

45 degrees

45 degrees

But if the sensor were turned 45 degrees, the green would be every other line. Does a Bayer sensor get less diagonal luma resolution than horizontal/vertical?

I think you take the H or V and times 0.707? Since the pixel pitch gives you about 0.707 for good MTF then it might measure 0.707*0.707*H=0.5*H with the OLPF and lens issues involved.

4K * 0.707 = 2.828K (square sensor?)

If you go past that point the aliasing and effects of the OLPF reduce the MTF to small values or erratic ones, depending on how sharp the lens is and the radius of the OLPF blur.

The Red and Blue resolution is not 2K since if you process the data right the green sensors see both red and blue, so your luma resolution holds up even for saturated subjects. The chroma resolution is lower for edges and small colored patches, but within colored areas the luma adds color brightness detail that would be less if you only interpolated the R G and B seperately. You should do the color matrix after the luma is sharpened then combined with the chroma to increase the detail in the saturated areas.

REDCODE may loose some resolution in the red and blue if they transmit it as 1/4 size color planes, but that is not a Bayer fault, but a way the Bayer data is sub-sampled. In a true raw camera the luma resolution can hold for all colors if processed right. As far as I know there are no Bayer filters that are strong enough to block all red and blue light from the green pixels. When you combine Luma with the red and blue chroma data you get full resolution details in the red and blue seperation images, it may be at a lower contrast since that depends on how the Luma is derived. In other words, the red and blue seperations are not made just from the red and blue pixels, and the interpolation of the blured image can give some sub-pixel details when true raw data is used since the interpolated images undergo un-sharp masking that can help locate the blur to a point between the pixels, then that is added to information from the green pixels, so although there may be some distortion, information can be derived for all points in all colors.

See the this answer to a similar question,

http://reduser.net/forum/showthread.php?p=416899#post416899

When you test a sensor you need to rotate the test chart to angles between 30 and 60 degrees because if you shoot with the sine wave pattern going vertically and pan the camera slowly you can get wild readings that vary from 2K to 4K depending on how the light and dark bands line up with the pixels in the sensor, if you rotate the test chart then the bands in the test chart cross more than one pixel so you can see the average better depending on how the aliasing is filtered out and how much sharpen in used. As you rotate the test chart you shoud read different peak values as you pan the camera around, highest at 0 and 90 degrees and maybe lowest at 45 degrees?

For making movies you need a resolution RGB 1280x720 in the print, so using a 4K Bayer filter can give you the needed resolution. Do not confuse resolution with pixel count, even film recorders and digital images in general need 1.414x as many pixels as the resolution required.

 

[Chris Davis]

Dan,
Many thanks for your time. I'm reading up on some of the concepts you discussed here and on your other post (that you had a link to), and will follow up later this weekend.
-Chris

 

[Chris Davis]

The Red and Blue resolution is not 2K since if you process the data right the green sensors see both red and blue, so your luma resolution holds up even for saturated subjects. The chroma resolution is lower for edges and small colored patches, but within colored areas the luma adds color brightness detail that would be less if you only interpolated the R G and B seperatel.

And here’s quote from the link you provided. I hope it’s OK to address it on this thread, is it’s directly related to the topic.
http://reduser.net/forum/showthread.php?p=416899#post416899

…Because the pixels are interpolated before being combined for the Luma you get some image details that are close to what you would get in a monochrome camera with the same number of pixels.

The main problem is getting clean chroma information since when you matrix the colors out since the signal is weak because the filters are weak you get a high noise level in individual pixels, to overcome that the chroma is averaged using the nearby pixels.

If the lens is sharp enough to just shine an image detail on one pixel than you can get false chroma for a small group of pixels since in order to get white balance a group of 4 pixels must see the same color light, no mater what color that may be. To overcome this chroma moire problem a OLPF filter is used to blur the light going to the sensor so that groups of four pixles, one red, one blue, and two green see more or less the same brightness and color of light.

As part of the de-Bayer - de-Moasic process the image is sharpened to counter balance the blur from the interpolation and the OLPF filter, this sharpen adds noise, chroma moire, aliasing, and so some low pass filtering is also needed to keep those issues low enough while at the same time letting enough sharpen take place to get a resolution reading that is close to the monochrome sensor value.

The Bayer sensor is just a monochrome sensor with a set of color filters over it.

And again from this thread:

…In other words, the red and blue seperations are not made just from the red and blue pixels, and the interpolation of the blured image can give some sub-pixel details when true raw data is used since the interpolated images undergo un-sharp masking that can help locate the blur to a point between the pixels, then that is added to information from the green pixels, so although there may be some distortion, information can be derived for all points in all colors…

I’m doing my best to make sense of this. My understanding of modulation transfer function and low pass filters is limited to some rather dense videos I watched in the above link. I know that an OLPF blurs the light in a specific way, and that a MTF “is a measure of how contrast is passing through the lens as it changes its special detail,” but I am hazy regarding how these apply in a practical way in a complex chain like on a digital camera.

When I read these posts, I hear at lest two basic ideas: (1) an optical low pass filter must “blur” the luma and chroma over a block of four pixels to get the same info all four pixels in the block and that (2) that a Bayer sensor can provide about as much luma res as it has pixels and that it can get a lot more luma res than just the blocks of four pixels would provide (using various methods you describe, but of which I have a little hard time following).

It would seem that, once a OLPF blurred the luma and chroma info across a block of four pixels, that group would turn into one piece of luma/chroma detail, and that it would be impossible to more luma/chroma resolution out of the individual pixels. Can you help me understand?

…Since the pixel pitch gives you about 0.707 for good MTF then it might measure 0.707*0.707*H=0.5*H with the OLPF and lens issues involved.

4K * 0.707 = 2.828K (square sensor?)…

Regarding the horizontal luma res, where does the number .707 come from? Are you saying that the diagonal res of a Bayer sensor may very well be half of the vertical/horizontal res? Would this be due to the way the green pixels are placed on the pattern?

Thanks for bearing with me on this.
-Chris

 

[Dan Hudgins]

No exact values for Bayer sensor/OLPF output...

No exact values for Bayer sensor/OLPF output...

Give up on the idea of finding an exact answer to the resolution of a Bayer sensor/OLPF+lens. It is not going to happen. It depends on the sharpen values used and other adjustments to the de-moasic code.

You can get some gidelines, but the combination of the factors that blur the image, the interpolation, and the sharpen/un-sharp mask and other processing lead to - and + factors.

The net result is the combination of many + and - factors, so there is no exact answer other than you can maybe get 80% MTF at 25% of the pixel dimension, and the MTF goes down from there as the frequencies get higher, at what point the MTF is equal to zero depends on the OLPF and other parts of the camera and lens.

The OLPF blurs "some" of the light but may not blur all of the light, so some detail may be there. If the OLFP was 100% efficient you would not get chroma moire, but you do so it helps but is not perfect, so you use the OLPF to reduce the chroma moire and then sharpen to reverse its effect on the luma as much as you can without making the aliasing too bad.

0.707 is the inverse of the square root of two, or how you find the diagonal, i.e. the hypotenuse of each pixel. Because lenses do not focus to a pin point and digital images alias, the net resolution is about 0.707 the pixel count to get a moderate MTF, then you take into account the diagonal factor and you get 0.7*0.7=0.5 about. The measured MTF depends on tha amount of un-sharp mask used to sharpen, you can push the MTF up but you get more chroma moire and aliasing and noise, so video cameras look bad because they use a sensor with too few pixels, and too much sharpen, to get more of a film look you use more pixels and less sharpen. Also to reduce aliasing the smallest image point should be about 1.414 to 1.5 times as big as a pixel so that part of the light falls on two or more pixels to give anti aliasing effects.

Luma should use ALL the pixels, so for uncolored subjects if the light is balanced on the sensor to mate with the Bayer filter's bias you get the same result as a monochrome sensor (one using a OLPF).

As noted, in the RED ONE, things get much more complex since REDCODE introduces more steps, and sub-samples the data and converts it to wavelets, that introduce more chroma moire that needs to be filtered out, so the Bayer issues are not just what impacts the final images.

 

[Graeme Nattress]

As noted, in the RED ONE, things get much more complex since REDCODE introduces more steps, and sub-samples the data and converts it to wavelets, that introduce more chroma moire that needs to be filtered out, so the Bayer issues are not just what impacts the final images.

I appreciate your explanations here Dan - saves me a lot of typing for sure, but that is not quite correct.

REDCODE does indeed introduce more steps to the process, but does not induce chroma moire beyond what an uncompressed recording would.

Graeme

 

[Chris Davis]

Give up on the idea of finding an exact answer to the resolution of a Bayer sensor/OLPF+lens...

OK

The OLPF blurs "some" of the light but may not blur all of the light, so some detail may be there. If the OLFP was 100% efficient you would not get chroma moire, but you do so it helps but is not perfect, so you use the OLPF to reduce the chroma moire and then sharpen to reverse its effect on the luma as much as you can without making the aliasing too bad. .

There is only one OLPF in a system like this, right?

The measured MTF depends on tha amount of un-sharp mask used to sharpen, you can push the MTF up but you get more chroma moire and aliasing and noise, so video cameras look bad because they use a sensor with too few pixels, and too much sharpen, to get more of a film look you use more pixels and less sharpen. Also to reduce aliasing the smallest image point should be about 1.414 to 1.5 times as big as a pixel so that part of the light falls on two or more pixels to give anti aliasing effects.

Interesting. Thanks for your help here. On other video forums, most people do not want to get way into the technical “why” of things. Here, I can barely keep up. I think I’m getting at least most of what you’re saying, looking things up as I go along. Do you have any links that talk more about how OLPF’s and MTF work together in a in a Bayer sensor camera, and how interpolation is done?

As noted, in the RED ONE, things get much more complex since REDCODE introduces more steps, and sub-samples the data and converts it to wavelets, that introduce more chroma moire that needs to be filtered out, so the Bayer issues are not just what impacts the final images.

…REDCODE does indeed introduce more steps to the process, but does not induce chroma moire beyond what an uncompressed recording would.
Graeme

Dan or Graeme: Do either of you have any links on REDCODE, how it works, how it must be processed, and what kinds of compression artifacts it gives?

 

[Graeme Nattress]

REDCODE is a 12bit wavelet compression scheme that works on RAW camera sensor data. Other than that, we've not really said much more about it. It works, and works well.

Graeme

 

[Dan Hudgins]

Semantics?

Semantics?

I appreciate your explanations here Dan - saves me a lot of typing for sure, but that is not quite correct.

REDCODE does indeed introduce more steps to the process, but does not induce chroma moire beyond what an uncompressed recording would.

Graeme

I do not intend to use the wrong term for what the superimposed wavelets do, but since REDCODE is "unique" I used the closest term I could find to describe the chroma artifacts that REDCODE produces, which might be seen in some underexposed images where the bit depth drops so the wavelets have more Posterization.

From what I can make out in the images taken from R3D, you are storing the Red and Blue images as 1/4 size and interpolating the Green image in the camera and storing it full size.

Since the Wavelets seem four times as big and the block sizes seem four times as big in the Red and Blue images when you superimpose the three color planes chroma artifacts are produced by the wavelet induced brightness errors, which only in part get filtered out, and boosting the choma in post might make them more visable.

Under low K light the high noise in the blue may make more erratic wavelets in the blue, so more chroma aberrations could be induced.

People reading this can see the wavelets for themselves as they become part of the resulting images, however they should not think that such issues are a reason not to purchase a RED ONE camera. The RED ONE camera is a good compromise system that meets the needs for many filmmakers, rather than exceed needed requirements it gives satisfactory results.

What do you call the wavelet induced chroma artifacts? Please don't say they do not exist, any change to brightness in small details MUST result in shifts in the chroma for the pixels in that location in the image, and that includes your "slight noise reduction", the so-called noise is mixed with true chroma data since any brightness difference between the colored pixels results in chroma and in true raw sensor that difference is the result of subject brightness and noise, but in REDCODE the difference is the result of subject brightness, noise, noise filtering, wavelet induced brightness errors in the primary colors, and filtering of wavelett induced brightness errors.

If you say that wavelets do not induce brightness errors to the primary color planes (or YUV), and cannot explain why because it is a trade secret, that just sounds silly, since everyone knows that the wavelets cannot be accurate for all pixels to a full 12bits, and therefore must induce chroma would not be there if true RAW data was being processed. If you are good at filtering out the extra chroma aberrations that does not mean that they were not induced.

 

[Dan Hudgins]

Olpf

Olpf

There is only one OLPF in a system like this, right?

Link about Epson's new OLPF filter,

http://www.epson.co.jp/e/newsroom/2007/news_20070906.htm

http://www.epsontoyocom.co.jp/english/product/Optical/set08/olpf_4/index.html

Here is another type, they have some files you can read on their site,

http://www.optics-online.com/lpf.asp

==

Yes a single chip camera has a single OLPF, it needs to be larger than the sensor so that the edge rays from the lens do not get cut off.

Since these are about 2mm think they add negative spherical aberration when fast lenses are used and introduce some focus shift issues between working full open and stopped down, most lenses are not corrected for having such a thick plate between them and the focal plane. To combat the loss of focus the sharpen can be made stronger, but that adds noise etc.

 

[Graeme Nattress]

In a 4096x2304 image, the R and B data is 2048x1152, and the green is either 4096x1152 or 2048x2304 which-ever way you want to look at it. We don't subsample the data lower than that - it gets recorded at the very same size as that, so I'm unsure what you mean by 1/4 size. Everything is stored at the very same size that the sensor produces.

Graeme

 

[Dan Hudgins]

Size

Size

In a 4096x2304 image, the R and B data is 2048x1152, and the green is either 4096x1152 or 2048x2304 which-ever way you want to look at it. We don't subsample the data lower than that - it gets recorded at the very same size as that, so I'm unsure what you mean by 1/4 size. Everything is stored at the very same size that the sensor produces.

Graeme

If you interpolate the red and blue before you compress them as wavelets then the wavelets issues would be the same size and they are in the green image. So I was speaking of the pixel dimensions of the color plane images going into the wavelet compression.

If you interpolate all four colors you get four 4096x2304 images to compress rather than the smaller ones, i.e. two green, one red, and one blue. Then when you superimpose those images to get Luma the size of the wavelet artifacts is smaller if you increase the bandwidth to allow for the same compression as you use in the combined green image now.

If you are dropping the red and blue pixels from the green, and compressing a green 2048x2304 image then the wavelets will distort the data more than if you are compressing an interpolated 4096x2304 image bacause vertical lines would be made zig-zag.

 

[Graeme Nattress]

There's quite a bit of thought and effort that went into precisely how we compress the raw data into the wavelets. All I can say is that the raw data is compressed "as-is" with regards to pixel dimensions.

Graeme

 

[Ralph B.]

The way REDCODE works is obvious

The way REDCODE works is obvious

REDCODE is a 12bit wavelet compression scheme that works on RAW camera sensor data. Other than that, we've not really said much more about it. It works, and works well.

Graeme

The way REDCODE works is obvious...
Unless you're trying to reverse engineer REDCODE the answer is:

It's Magic!

 

[Chris Davis]

Optical Low Pass Filter = 2 pixels to make a line? How 3.2K from 4K?

Optical Low Pass Filter = 2 pixels to make a line? How 3.2K from 4K?

Give up on the idea of finding an exact answer to the resolution of a Bayer sensor/OLPF+lens. It is not going to happen. It depends on the sharpen values used and other adjustments to the de-moasic code.

You can get some gidelines, but the combination of the factors that blur the image, the interpolation, and the sharpen/un-sharp mask and other processing lead to - and + factors.

The net result is the combination of many + and - factors, so there is no exact answer other than you can maybe get 80% MTF at 25% of the pixel dimension, and the MTF goes down from there as the frequencies get higher, at what point the MTF is equal to zero depends on the OLPF and other parts of the camera and lens.

The OLPF blurs "some" of the light but may not blur all of the light, so some detail may be there. If the OLFP was 100% efficient you would not get chroma moire, but you do so it helps but is not perfect, so you use the OLPF to reduce the chroma moire and then sharpen to reverse its effect on the luma as much as you can without making the aliasing too bad...

The measured MTF depends on tha amount of un-sharp mask used to sharpen, you can push the MTF up but you get more chroma moire and aliasing and noise, so video cameras look bad because they use a sensor with too few pixels, and too much sharpen, to get more of a film look you use more pixels and less sharpen. Also to reduce aliasing the smallest image point should be about 1.414 to 1.5 times as big as a pixel so that part of the light falls on two or more pixels to give anti aliasing effects.

...In the end, you're always getting at least 2k resolution on R and B, and you can see detail "just" out all the way to 4k in luma / green, but we call it at around 3.2k, which is obviously significantly higher than any video camera...

:anti-oldI have a new question regarding an old thread. I would like to observe forum etiquette, so please kindly inform me if it is better to start a new thread in this kind of situation.)

1. Chroma resolution aside, it would seem that an Optical Low Pass Filter would limit luma resolution to half the number of pixels (1/2 horizontally or vertically, or 1/4 the area of pixels). If an OLPF is about 1.4-1.5 times the width of a pixel (is this correct?), it would seem that all light/information would be "required" to activate/touch at least 2 pixels (horizontal or vertical), or 4 pixels (in area).

This would mean that the horizontal resolution of the original 4K RED One could not exceed 2K, which is not the case. The original RED One sensor could resolve 3.2K lines of horizontal resolution with about 4K of horizontal pixels. (Note: I am speaking of luma resolution only, and am speaking in lines, not line pairs). What am I missing?

Thank you for your time. I'll try to check this thread, but may not be able to reply for a couple of days.