Is R3D 4:4:4 or 4:2:2?

[Roberto Lequeux]

Love this thread. This stuff is pretty over my head but I can hang enough to be curious and want to learn more. Now I understand what you were talking about the other day Bruce, mostly anyway.

If with a full resolution but slightly more lossy compression applied only to chroma you churn out files exactly the same size as those with far lower chroma resolution -- you have a far better image at no cost!!

Why hasn't this happened yet? (aside from our inability to clone Graeme) Would it be heavier on processing than sub-sampling? What's the deal?

[conrad gaunt]

My main issue here is correct use of terminology, because there's been some serious abuses of this terminology in the past from people who should know better.

Graeme

I know you're a stickler.. and I'm guilty

I've never computed accurate numbers myself, but it wouldn't surprise me if some of the RGB->YUV->RGB transformations through out as much or more than 50% (i've heard higher numbers) of the information originally present, so it would be wrong to compare 4:2:2 yuv signals to a raw RGB sensor output .. such as redcode

Until someone invents some kind of multi-layered full resolution RGB sensor, I predict we're going to be using the 4:2:2 RGB "ratio" for a while.. and with ultra-res and downsampling, its hard to imagine a better system right now ..haven't seen one

People use terminology "wrongly" all the time. My pet hate was the guys from fxguide RC, early on, who for a long while (but not for a long time now) would refer to "doing a half/quarter debayer".. more accurate to say, "doing a partial inverse wavelet transformation" (which is of course the reason we can read back only part of a R3D and still get an image out, albeit at some lower resolution, and the debayer resolution incidental).. but I got their points

[conrad gaunt]

Love this thread. This stuff is pretty over my head but I can hang enough to be curious and want to learn more. Now I understand what you were talking about the other day Bruce, mostly anyway.

If with a full resolution but slightly more lossy compression applied only to chroma you churn out files exactly the same size as those with far lower chroma resolution -- you have a far better image at no cost!!

Why hasn't this happened yet? (aside from our inability to clone Graeme) Would it be heavier on processing than sub-sampling? What's the deal?

My answer would be long. Here's my short version.

Cine used film, job done. TV only needed to meet the broadcast signal standard..SD.. which tended to use various myriad forms of YUV over time, now we have more widespread use of codecs and higher resolution digital viewing options, and its happened hand in hand with internet speeds and computers, quite quickly compared to the evolution of say motor cars which are surprisingly similar to the first cars produced, as are production techniques in many ways.. and so we find our selves asking "why hasn't this happened yet?" .. right now, what is the resolution standard 720p, 1080p , 4k -> im still on SD for viewing or monitor? stereo? .. and the distribution format? .. I don't know, and everyone else is guessing, including the manufacturers who are at the mercy of "what people are actually going to buy".. so my answer is, all sorts is happening, at different speeds, sometimes in secret, sometime not, probably right now, maybe! Wish I still had my sisters of mercy live at the royal albert hall laser disc..

..difference is, Ford knew he had to make cars his employees could afford to buy, and aspire to buy .. camera manufacturers seem to have a different idea.. which is why i dropped out of "camera owning world" for ten years, and wholeheartedly, and totally have skipped HD.. wish I could skip owning an mx sensor frankly, but you eat what you're given or starve

[Les Dittert]

Conrad, what is it that you are trying to do?
For me it's getting the max image quality from the gathered data stored in the r3d.
My pet peeve right now is how the debayer rendering from RCX is looking.
It's a linear debayer as far as I can tell.
We have done our own r3d decode and saved the result in a Cineform RAW mov. When I set the debayer to linear in Firstlight, Cineforms control program, it looks exactly like the RCX debayer. Soft.
I prefer the more detailed debayer preformed by the rocket card. When I set the debayer to other versions, I can get to the detail level seen in rocket card renderings. The slight difference I see may be due to the re-compression of the RGGB once again. Or maybe we have a math error still, in our code.
For the less than tech savy, no, we did not just transcode RCX output as a Cineform RGB. We are taking the bayer data and compressing that, preserving the RAW nature of the image.

[Uli Plank]

I'm under the impression that you look at things from two very different perspectives. Conrad takes the information theory POV, where it's very clear that a Bayer pattern sensor can't resolve the same amount of information for all colors. Period. (BTW, I knew that an OLPF is filtering both luma and chroma).

Graeme is taking the POV of the mastermind behind RED's debayering/compression methods, who is achieving the best cinematic images to be looked at in the end. What I'm getting from his explanations is this: as long as you can use the RAW information from a Bayer pattern sensor, you can optimize the process in a way that can't be achieved after any YUV subsampling because you have a whole color space. Take a pure blue vs. a bluish-green for example, the pure blue can't be reconstructed to the same scene detail the mixed color could reveal, since there is more green in it. This would be true for any color other than blue and red. But even then you'd have dyes which are not perfect in filtering plus light scatter from neighboring photo sites and so on. If you take this into consideration, it's far more complex but also more capable in the right hands.
YUV subsampling is a pretty harsh process in comparison.

[conrad gaunt]

I'm under the impression that you look at things from two very different perspectives. Conrad takes the information theory POV, where it's very clear that a Bayer pattern sensor can't resolve the same amount of information for all colors. Period. (BTW, I knew that an OLPF is filtering both luma and chroma).

Graeme is taking the POV of the mastermind behind RED's debayering/compression methods, who is achieving the best cinematic images to be looked at in the end. What I'm getting from his explanations is this: as long as you can use the RAW information from a Bayer pattern sensor, you can optimize the process in a way that can't be achieved after any YUV subsampling because you have a whole color space. Take a pure blue vs. a bluish-green for example, the pure blue can't be reconstructed to the same scene detail the mixed color could reveal, since there is more green in it. This would be true for any color other than blue and red. But even then you'd have dyes which are not perfect in filtering plus light scatter from neighboring photo sites and so on. If you take this into consideration, it's far more complex but also more capable in the right hands.
YUV subsampling is a pretty harsh process in comparison.

I think my thinking probably covers a bit more than information theory, it involves practical programming of wavelets, debayering algorithms and many other techniques including fractal compression (my own kind), 3d visualisation/ raytracing, image resampling (my way), and recently audio dsp..along with analysis of r3d files and about 6000 other things. I started programming 30 years ago, 31 soon. How you perceive my POV, is not very accurate..but not offensive

My primary interests are non-realtime visualisation technology, and image pipelines..im a patient man

I bet I could explain a lot of what makes a red frame look so nice, and simulate it ..but its amazing the compression rates you can get if you just stick 16bit raw data thats had wavelet transformations applied, into a png or j2k container.. but when you start doing non-realtime stuff, like using wavelet coefficients to guide super-resolution resampling..debayering and wavelets join forces.. i spent a while on that a couple of years ago

my focus at the moment is lossless D.I compression, something I've explored at several different times over twenty years, but its more relevant than ever for me right now

Raw is great for acquisition, but you need something more 4:4:4 (in nature) to archive masters, and sizes then get funky of course

My problem is, im not a commercial progammer, im an explorer

anyway..is Rob Lohman not the main codec brain, lets not forget the other elves!

[Eric Z]

Graeme,
Any chance the new Red Dragon sensor will be a Foveon sensor, like in the Sigma SD1 (15MP x 3 layers)?
That would surely help keep 5K footage at 5K, without any need to debayer it to lower resolutions.
I realize the Foveon sensor introduces its own problems, such as less and less light for the bottom 2 layers, but all in all - its sounds like a very good solution.

Any opinion on that?

[conrad gaunt]

Conrad, what is it that you are trying to do?
For me it's getting the max image quality from the gathered data stored in the r3d.
My pet peeve right now is how the debayer rendering from RCX is looking.
It's a linear debayer as far as I can tell.
We have done our own r3d decode and saved the result in a Cineform RAW mov. When I set the debayer to linear in Firstlight, Cineforms control program, it looks exactly like the RCX debayer. Soft.
I prefer the more detailed debayer preformed by the rocket card. When I set the debayer to other versions, I can get to the detail level seen in rocket card renderings. The slight difference I see may be due to the re-compression of the RGGB once again. Or maybe we have a math error still, in our code.
For the less than tech savy, no, we did not just transcode RCX output as a Cineform RGB. We are taking the bayer data and compressing that, preserving the RAW nature of the image.

same thing as you by the look of it

there are some very good linear interpolation based debayering algorithms actually, their secret is they're adaptive. Microsoft wrote a paper about one of theirs which did well against the well known "Variable Number Of Gradients" algorithm, which is great for natural scenery..

I quite like reds debayer, and cineforms.. but I tend to archive downsampled masters, so that vastly reduces aliasing or artifacts and makes all good anyway.. I say that, I haven't for three years because my money is not coming from post production at the moment

The secret of Reds image IMO is the balance they've got between softness and sharpness. I'd rather have high resolution that is a little soft, than a high (or low) resolution image with aliasing, and thats perhaps a lot to do with OLPF, and partly to do with wavelet transformations throwing away small details similar to noise reduction systems (some of which are wavelet based of course), and partly because debayering not only re-creates missing pixels, but often blends adjacent ones.. which is why an unsharp mask filter may be your friend..if thats your thing. A lot of video cameras with baked in looks used to aim for high acutance, because the human visual system treats it like real-resolution, but in reality its a sock pushed down the front of the cameras pants (thats a metaphor, I do know real cameras are always in the nude), but no real meat

I like using my own downsampling software, which is both slow and bullet proof in equal measure, and can do 28k -> SD conversion, of even awkward frames.. its linear interpolation, with gaussian weighting, and comes in 32bit, 64bit and 80bit versions.. I will produce a commercial grade version when I eventually buy a new compiler (don't ask..) ,

[conrad gaunt]

Graeme,
Any chance the new Red Dragon sensor will be a Foveon sensor, like in the Sigma SD1 (15MP x 3 layers)?
That would surely help keep 5K footage at 5K, without any need to debayer it to lower resolutions.
I realize the Foveon sensor introduces its own problems, such as less and less light for the bottom 2 layers, but all in all - its sounds like a very good solution.

Any opinion on that?

You don't debayer to a lower resolution, you always debayer at full resolution, then downsample to a lower one..

..or, you can perform a partial inverse wavelet transformation of the r3d, to whatever resolution level you decide (in this example, a lower resolution), followed by a debayer.. which is not as good, but faster than above method

btw, foveon sensors are a bit crap at the moment,.. but i might be tempted by a light field sensor in the future , from a documentary POV

[Graeme Nattress]

Long Answer: Foveon has some real colorimetry issues - what you save on not having to demosaic, you spend on having to build in extensive noise reduction as part of the basic decode process (see the Foveon white papers). This is because silicon is a pretty poor colour filter. It also makes it very poor at higher ISO and leads to other imaging issues that various forums have pointed out, especially in the new chip with the smaller pixels.

Most of the Foveon fans like excessive aliasing which adds fake sharpness to their images. We think such levels of aliasing are unacceptable for cinematography use. When used with a proper optical low pass filter the "benefits" of Foveon have diminished so far as to make it pointless. Then you're stuck with a sensor that's not fast enough to do motion at 120fps (I dunno what it's fps is, but it's no-where-near where our sensors are).

Short answer: no.

Graeme