Is R3D 4:4:4 or 4:2:2?

[Uli Plank]

Speaking of hexagonal designs, there is one in the photographic world: http://www.luminous-landscape.com/reviews/cameras/s3-pro.shtml

They use a combination of small and large photosites to get more DR, but I doubt this would work for motion. Imagine the projection of a very small, highly contrasty object (a far away car's headlight for example) moving over those photo sites, I reckon you'd see some amount of flickering.

 

[Sanjin Jukic]

14 stops of DR for any available cinema digital camera is still sort of marketing blow up value

that was maybe possible to achieve in a kind of ideal lab environment but actually never in a real world.

But film stock has already proven example of 14.5 stops of DR.

Have a look below at the DR test examples done by the SCCE on Alexa, RED ONE MX and KODAK 5219 film stock.

Click at the image to get enlarged.

 

[Graeme Nattress]

A. downsampling increases DR and reduces noise. their quoted "double the resolution" helps a lot with that going down to a 4k.

Except there's no downsampling going on at all for them to achieve 4k. That is why they use that particular arrangement - it's a straight sample on green, and some simple interpolation for red and blue.

Graeme

 

[Wayne Morellini]

Guys I'm ashamed. What a load of old cobblers before.

The only advantage that I see with the photo site diagonal arrangement is the possibility of faster read out time perhaps for more broadcast oriented applications.

Are they really still peddling “new math” to sell the cameras?


I suspect they were originally worried that they went with photosites oriented at 45 degrees that the customers/punters would not like it or be able to get their heads around it as everybody knows “pixels” HAVE to be square and upright (right?). I think they devised this sort of marketing scheme to get people to go “AHHAAA” we must use 45 degree oriented photosites because it gives you MORE… MORE! (we want more, we always want more!).

Eric

Or maybe something like this:

www.reduser.net/forum/showthread.php?71931-Is-R3D-4-4-4-or-4-2-2&p=949198&viewfull=1

I can tell you from decades of calibrating CRT projectors that regardless of how high the chroma resolution of your image is sampled, blue will still look softer than red and red softer than green due to the sampling limitations of human vision.
There are many valid reasons for the virtually universal acceptance of Bayer CFA sensors in digital imaging. It remains the closest and most efficient mechanical mimic of human visual perception to date. The filter dyes are not pure RGB, but modified to simulate the way the eye actually separates color information and the relative color sample ratios also mimic human color resolution discrimination. Nobody much complained about this technology until it was applied to motion cameras. Many other sensor formats and designs have been tried, but non has come close to being as successful for visually relevant results.

Mimicking the human eye mechanisms ultimately is a bad way to capture visually good images, in that it is miminalistic, and requires the veiwer ti be onna complete, and disatisfactory, roller coster ride, where thy can not look or focus anywhere else or adjust their own pupils. You sample more completely and accurately to get over the misalighnment of such system to each audience member's individual visual system, it's individual state at the time and peculiarities, as well as the direction they are focused on.

Early debayering might make nice pictures, but was less suitable for moving images, and as I will show here, a lot a pro Bayer arguments are smoke and mirrors. If you viewed with a human eye into a camera you would see distorted almost fish eye like view, with rapidly decreasing resolution to the outside, no blue in the middle, which you pickup when you flick your eyes around many times, using different parts of the retina, as well as scannihg detail, to see the image faster or finer with different color fidelity, and an amount of aberrations, that your brain all tries to filter to make it look normal. Put something artificial in front, and your brain is not controlling it to look natural. It will pickup missalignments with reality outside it's own visual system more than nothing.

If Bayer was such a good effective analog to the human eye to use, then why debayer to make it more 4:4:4 like, the eye would fill in missing detail and color, aliasing, and miore, but it doesn't fill pictures of real life in that way so much, because life is only 4:4:4+. To fill in what the eye is looking for too fool it, we need 4:4:4 of sufficient fill, color and resolution, until the brain is happy it is seeing something authentic. Simple, period, and I can bewilderingly believe people here are believing the stuff peddled.

Calculations are done on discrete dimensions of points, mathematically this adds up to an array of squares for full efficient used coverage, not misaligned rows of packed circles (hexagons, which Eric might appreciate I just realised is probably directly in rekatiin to how many dimensions there are in the universe etc (just realised this is getting close to my unpublished systems theory of physics)). In reality such a hexagon system needs calcukatiin to rectify it, but the smaller you go the less it is going to be noticed. Again, 4:4:4 hexagons are the go until you get small enough, so the eye does not discern individual primary colored pixels outside of their other color components as the eye's visual elements edges over and around them. It should be evident, that most all endeavours in sensor processing is to get around elements referred to here.

So 4:4:4 is the sensible option, even Bayer has to try to emulate it. Bayer is a quick and convent fake that gives nice results. Make a layer sensor properly (well), you have three times more data, but such a better picture. The alternative is too take resolution down further than 4:4:4 needs, or interpolate the correct missing data, but are those prices.

 

[Wayne Morellini]

Absolutely… it is well known in the field of so called Psycho-physics that small blue objects (especially at a distance) are hard to discern. This is entirely down to the composition of the Fovea/macula in the human retina of course.

Basically the S-cones in the retina responsible for detecting short wavelengths (i.e. blue) are much more widely spaced in the retina than the L and M cones. One of the reasons cited for this is that axial chromatic aberration of the human lens blurs the short wavelength portion of the retinal image. Therefore don’t evolve a density of blue sensitive cones that can’t be resolved anyway…. (right?).


There are a couple of ‘juicy” little facts about that.

1. In the very centre of the visual field in the central Fovea there appear to be NO short wavelength cones at all!

2. Secondly that this “spatial” defocus (of the blue/S cones) causes the short wavelength portions of an image to “code” neurologically slower temporal variations than L- and M cones especially for dynamic imagery. I.e. it’s a slower more blurry response and perception in the “blues”.

I think that might be an interesting thing to take advantage of when designing compression algorithms for dynamic sequences.

With the Bayer pattern, At least with the blue (of course the green is well taken care of) having a spatially offset blue “receptor” that has a more spatially generalized influence is completely consistent with how the human visual system operates. Actually it’s better than the fovea of the human eye… (I think).


E

See above. The blue focus problem for cameras is outlined before to.

I wonder if this center of vision lack of blue is responsible for the integration of color at such low resolutions (150dpi by my tests) when you scan an image, the spot is large enough.

Hrvoje, Which is deliberately drawn with big gaps to make it look worse? Why are square pixels drawn as circles - perhaps to disguise that in reality, they're exactly the same - one just rotated through 45 degrees and uses pixels with half the area.

Graeme

Suspect I know, but have you considered they might be a mathematical point sampling as circles , and the relative density might indicate the increased pixel density over a pure 4k Bayer sampling. Then why not just increase the resolution of the Bayer sensor the same. It is not invalid, Epic uses 5k to produce better 4k.

Have not looked into it properly, but there are 25 Bayer pixels to 41 whatever they call them pixels.

You are right, the arrangement is rectilinear and 45° rotation does not make it hexagonal, but the first published visualization of the pixel arrangement, in case it was not intentionally misleading, gave some clues in the principle in which the values are combined into one pixel.

Not sure about the pixel proportions, but if I recall correctly the principle was something like this:

If the values are combined in the way this visualization suggests, i.e. subdivided R&B pixels with cross diagonal combination, it gives a room to assume the principle is not just based on simple 45° rotation, but the different DeBayer logic as well, meaning not zig-zag.
Meaning symmetrical spacial offset of R&B to G.

www.reduser.net/forum/showthread.php?71931-Is-R3D-4-4-4-or-4-2-2&p=949198&viewfull=1

As also was noted latter, my examination showed there are many ways to devide up the pattern. Using interpolation of four surrounding pixels in the position where four corners meet you can get your extra pixel resolution towards 8k. As I think I noted sometime ago, and I forget if one of you guys picked up on it, if you have one on one by sized, you get a hexagon with small corners. So various alignments give you various shapes and options specifically due to rotation, guys this is defintely not Bayer as you are thinking before.

First published visualization of F65 sensor, showed a 45° rotation with additional vertical/horizontal divisions, affecting only R&B pixels, if I recall correctly. Not sure if that was visualized exactly the way it is actually done. At this point it is just guesswork.

This rotation approach was taken from ClearVid CMOS consumer technology Sony introduced in 2006. Back then they used more percent of green photosites.
Here is the interview mentioning this technology. Visualization of those sensors differs from the F65 sensor in the amount of green photosites and lack of additional divisions, but the principle seems similar:

http://www.camcorderinfo.com/content/The-ClearVID-CMOS-Sonys-Chip-of-Choice.htm

Some experts here should have picked this up quickly.

That diagram would help, particulary if the pixel has subunit addressing. But anybody can simply put a square array of he right sized square rotated over another right proportiined normal graphic if they want.

Aliasing problems go away of you are doing super debayer, that is debayer temporally.
So far all the talk is derived from still photo processing. Those papers are easy to find
We have motion, which does add significant sampling information that can be recovered.
I will present samples soon. I have RGBG planes to work with, avoiding the current Red methods.

Covered that before, see interframe debayering.

 

[Wayne Morellini]

This visualization shows two options with differences in pixel size, simulating pixels arrangement with micro lenses.
Space between pixels on higher resolution examples is unintentionally smaller, as it was faster to just scale the first reference visualization I made, so this is merely a rough comparison.

First row from the left - resolution starting point, next simple double density, next same double density rotated 45°
Second row - starting point, next 45° rotation & proportions as Sony previously visualized, next pixel size comparison of that approach

In case the visualization previously published by Sony does not suggest additional pixel sub-divisions at all,
the point was to allow the analysis of this approach in general, regarding chroma values per same surface, and demosaic options.

At least of the principle...before the actual pixel-peeping.
For those who care that much with the level of picture quality which Red cameras achieve today.

Thanks for your responsibility Hrvoje. Visualizations in my head are taxing, even though I have to use them. Apparently some people have problems seeing it, stuck in what they already know. It is merely a fact rotated by 45% on top of another fact to create a newer fact, but some people have problems distiguishing concepts as facts or putting them together to make new ones. In this way people can start seeing it before they even visualise it.

 

[Eric Lange]

This US patent Application filed in 2007 (from Japan) describes a very particular set of solutions to problems normally associated with diagonally arranged photo-sites on a CMOS imaging array.
See

http://www.google.com/patents?id=xV-iAAAAEBAJ&printsec=abstract&zoom=4#v=onepage&q&f=false

You can click on the full citation and I urge folks to look through all of the diagrams at very least.

This might answer a few questions and also put a few things to bed… until physical testing is possible.

From Summary of the Invention:

“As described above, lines in a CMOS image sensor are arranged in the vertical and horizontal directions with respect to the chip or image pickup area. However, when miniaturization of pixels is promoted, or when a pixel transistor is shared by a plurality of photo diodes, light incident on each photodiode may be obstructed, particularly in a CMOS image sensor having a diagonal pixel array. In addition, vertical and horizontal lines (for example, the vertical lines 16 and 17 in FIG 23) limit wiring layout possibilities, and thus the area occupied by photodiodes has to be reduced.

Con’t “[0013] The present invention has been made in view of the above circumstances. Accordingly, there is a need for a solid-sate pickup device with a diagonal pixel array which prevents obstruction of light incident on a photoelectric conversion unit and thus allows an increase in a light-recieveing aperture ratio of photoelectric conversion unit.…

…
[0015]… “With this arrangement, obstruction of light collection of photoelectric conversion unit can be prevented.”

I.e. for diagonally arranged photo detectors.

This document also cites other variants of diagonally arranged arrays, but this one claims to solve more of the problems normally associated with diagonal arrays.

Clearly, the way these things are actually physically manifested as very complex layered architectures and shapes of photo sites and wiring are about a million miles away from the near infantile marketing materials and diagrams that one normally finds.

What I’m saying here , even if you were to three dimensionally re-construct everything presented in the above patent application in a very high end multi-physics simulation software there is still no way that one would be able to actually extrapolate the actual performance of the system described. Really!

I would gently point out perhaps that maybe we should desist from extrapolating very complex solid state quantum behaviors, DSP and performance from lousy marketing materials.
It’s terrific that everybody here is super smart and very interested and I too have been guilty when the F65 was first announced to also sketch out on the back of an envelope the expected characteristics of the F65… but in reality this is a bit of a folly, but never the less is good to get some sort of appreciation of how these things might “tick”.


In this sense I feel for Graeme somewhat that he has to very patiently field such queries and dispel potential FUD from other marketing claims. That’s a lot of hours taken out of his day and I guess such marketing from other quarters can become a little irritating after a while.

I am very grateful to Graeme over the years to kindly educate me on this and that and gain a greater appreciation of almost a more “holographic” understanding of the interaction between various arrays and sampling strategies as being a much more complex and subtle macroscopic process. He has changed my perception a great deal not to think of photosites in such a “discrete way”. Thanks for that!

Maybe I foresee a “thread closed” in the not too distant future just so Greame can get on with his day job! (he, he).

So I eagerly await an input image and output image set of tests also for the F65… it’s awesome there is community that can pull together to perform such test. Overall its great (I think) that the whole 4k ecosystem is gathering momentum and investment for the long haul…. Every player is important to help make that happen including RED’s (supposed competitors).

Still I don’t know what 4:4:4: means anymore? … and is it a useful idea?

Cheers,

Eric

Foot note, I'm not saying this is how exactly the F65 functions but is used as an example to illustrate the true complexity of such a sensor and that it is not possible to extrapolate performance from marketing diagrams or even the best information available of the true architecture of such a system.

 

[Tom.Wong]

but just out of curiosity are you saying if red MX sensor was an 8k sensor and down sampled to the 4.5k then maybe it would have higher DR etc?

it would definitely reduce noise, but there are a lot of other factors for DR. That's more actual sensor design and processing of what's coming off the sensor. If the sensor actually isn't "seeing" 14 stops than their is no 14 stops. and if the sensor is seeing 14 stops, but the sensor data being processed doesn't hold all 14 stops, than again, no 14 stops. If RED made an 8k sensor just by packing more pixels into the same one they have now, all it would mean is more resolution. downsampling from 4k with that will reduce the noise, but there are so many factors involved that's not the end all be all for it.

plus the camera will probably be bigger, generate a ton more heat, you won't get the same framerates, etc. etc. all I mentioned are simple principles that are the baseline for fundamentals of the image capture, but there are always tricks to work around it, and it's not always set in stone. If someone designed a true 8k sensor but has a poorly made sensor and processing to back it up, it could be even noisier and have less DR than anything else out there.



but back on track, i still dont get why people are asking if redcode is 444 or not. chroma subsampling doesn't apply to RAW. however much color the sensor captures is just that. I've mentioned it before, and this is probably the 500th thread i've seen on this over the past year and half alone.

chrom sub sampling was designed in the broadcast world of basically cutting down the bandwidth needed for streaming or airing footage. the human eye is less sensitive to chroma values than it is to luminance and resolution. so that was a shortcut made to reduce the amount of information that was being captures. 444 in simple terms, is just the maximum chroma values that can come off the sensor.

but if the sensor isn't capturing accurately, it doesn't mean you can't get 444 out of it. you can get ALL the color that's coming off the sensor in the container, doesn't mean it's any better or worse. RAW is all the sensor information inside of a file that you can manipulate in post, using your computer to process the image manipulating rather than having the camera figure it out. and thus, it holds all the color information the camera has to offer.


even a handycam can be 444 if you wanted it to be if you ripped out the sensor and put it in a body that would re process the information and you capture it at 444. that quality isn't gonna change THAT much.

but the term is tossed around way too much because it also locks the color values in a certain place, and is just so generally used as the way to say that the capture format has the most color information. fact of the matter is, we are beyond these kind of terms and look to much higher ground to accurately see how much color a camera offers.

 

[Eric Lange]

it would definitely reduce noise, but there are a lot of other factors for DR. That's more actual sensor design and processing of what's coming off the sensor. If the sensor actually isn't "seeing" 14 stops than their is no 14 stops. and if the sensor is seeing 14 stops, but the sensor data being processed doesn't hold all 14 stops, than again, no 14 stops. If RED made an 8k sensor just by packing more pixels into the same one they have now, all it would mean is more resolution. downsampling from 4k with that will reduce the noise, but there are so many factors involved that's not the end all be all for it.

plus the camera will probably be bigger, generate a ton more heat, you won't get the same framerates, etc. etc. all I mentioned are simple principles that are the baseline for fundamentals of the image capture, but there are always tricks to work around it, and it's not always set in stone. If someone designed a true 8k sensor but has a poorly made sensor and processing to back it up, it could be even noisier and have less DR than anything else out there.



but back on track, i still dont get why people are asking if redcode is 444 or not. chroma subsampling doesn't apply to RAW. however much color the sensor captures is just that. I've mentioned it before, and this is probably the 500th thread i've seen on this over the past year and half alone.

chrom sub sampling was designed in the broadcast world of basically cutting down the bandwidth needed for streaming or airing footage. the human eye is less sensitive to chroma values than it is to luminance and resolution. so that was a shortcut made to reduce the amount of information that was being captures. 444 in simple terms, is just the maximum chroma values that can come off the sensor.

but if the sensor isn't capturing accurately, it doesn't mean you can't get 444 out of it. you can get ALL the color that's coming off the sensor in the container, doesn't mean it's any better or worse. RAW is all the sensor information inside of a file that you can manipulate in post, using your computer to process the image manipulating rather than having the camera figure it out. and thus, it holds all the color information the camera has to offer.


even a handycam can be 444 if you wanted it to be if you ripped out the sensor and put it in a body that would re process the information and you capture it at 444. that quality isn't gonna change THAT much.

but the term is tossed around way too much because it also locks the color values in a certain place, and is just so generally used as the way to say that the capture format has the most color information. fact of the matter is, we are beyond these kind of terms and look to much higher ground to accurately see how much color a camera offers.

Tom thanks for that!... nicely set out and very clear… really nice explanation.

But for a moment there I was scratching my head thinking how does (macroscopically) the number of K improve dynamic range (again) huh???[Have I missed something???]

I have to say REDUSER from time to time does have one questioning what one knows… which is good.’

Cheers,

Eric

 

[Tom.Wong]

I might have been a little misleading, my bad. downsampling won't increase DR, it will lower the noise in the image. the direct correlation with DR is what the camera physically captures, and the container it goes into. bit depth etc. sensor design and processing is the bread and butter for DR.

 

[Eric Lange]

I might have been a little misleading, my bad. downsampling won't increase DR, it will lower the noise in the image. the direct correlation with DR is what the camera physically captures, and the container it goes into. bit depth etc. sensor design and processing is the bread and butter for DR.

No, no, not at all you set it out very clearly, I was referring to what in essence you were responding to where another poster had put forward the idea that down sampling would have a useful impact on Dynamic Range. I though you put that to bed quite nicely…

No confusion just my rather indirect comment.

Cheers,

Eric

 

[Hrvoje Simic]

Thinking out loud...

Potential direction in imaging and display technologies.

 

[Eric Lange]

Thinking out loud...

Potential direction in imaging and display technology.

That’s cool… I would make the greens bigger and the blues smaller than the reds…

[Don't say that's the same as Q67... ].

Don’t ask me how to etch that circuitry onto silicon.

Just to throw some more onto the tin foil hat paranoic bonfire… how do you know that your RED MX sensor is not already arranged on diagonal boundaries …? (only kidding).. you don’t want people pulling their cameras apart and putting them under a microscope just to make sure… the point being it dosen’t matter.

Target => output image.

Nice One,

Eric

 

[Hrvoje Simic]

That’s cool… I would make the greens bigger and the blues smaller than the reds…

I understand the basis for the suggestion, but that would break the hexagonal balance of the array.

[Don't say that's the same as Q67... ].

I won't.

Don’t ask me how to etch that circuitry onto silicon.

Quite a task probably.

Just to throw some more onto the tin foil hat paranoic bonfire… how do you know that your RED MX sensor is not already arranged on diagonal boundaries …?

Don't really care. Red imagery rocks regardless.

Honeycomb structure is not just about the diagonal arrangement. It has 3 axis spread, not just 2.
Also, distances between centers of R-G-B cells are the same.

On a square grid centers of R, G & B form a right isosceles triangle. In hexagonal they form an equilateral triangle.
Meaning balance.

Hexagon allows larger coverage of the micro lens than a square does. Also on RGBW example on the left, "main" hexagonal pixels
consist of RGB + additional luma pixel with R-G-B in pairs surrounding it in perfect balance.

&

Aliasing guesswork on 2 vs. 3 axis spread.

 

[Evin Grant]

Thinking out loud...

Potential direction in imaging and display technologies.

What I don't understand is what the white centers represent?

 

[Peter Moretti]

I think white is no color filter at all.

 

[Peter Moretti]

FWIW, personally, I think the real advancements in sensor design will come from increasing fill factor, read rates, image processing algorithms and by tuning/eliminating the OLPF. I don't think the pixel shape or arrangement is going to amount to much, except if it can help reduce aliasing and moire. But of course I could be very wrong, LOL!

 

[Hrvoje Simic]

What I don't understand is what the white centers represent?

White sub-pixel.
Photosite dedicated to capture pure luma.

RGBW approach is already used in recent display tech and sensor tech.
Fuji X1 Pro camera has a RGBW LCD. Samsung Galaxy S smartphone has a RGBW OLED.
LG TV announced on CES 2012 has a RGBW OLED. Sony recently announced a RGBW sensor.

 

[Peter Moretti]

I do like the idea of white b/c it has no loss from the the color filter, so it probably makes a better choice than let's say yellow.

 

[Eric Lange]

I understand the basis for the suggestion, but that would break the hexagonal balance of the array.



I won't.


Quite a task probably.



Don't really care. Red imagery rocks regardless.

Honeycomb structure is not just about the diagonal arrangement. It has 3 axis spread, not just 2.
Also, distances between centers of R-G-B cells are the same.

On a square grid centers of R, G & B form a right isosceles triangle. In hexagonal they form an equilateral triangle.
Meaning balance.

Hexagon allows larger coverage of the micro lens than a square does. Also on RGBW example on the left, "main" hexagonal pixels
consist of RGB + additional luma pixel with R-G-B in pairs surrounding it in perfect balance.

&

Aliasing guesswork on 2 vs. 3 axis spread.

Hrvoje check this out...

http://www.google.com/patents?id=-oyUAAAAEBAJ&printsec=abstract&zoom=4#v=onepage&q&f=false

So Hewlett Packard submitted a US patent application in 2002 on the same/similar idea, (at least the left hand depiction).

The patent is very simple and they talk about how to de-mosaic a tri-laterally symmetric array as well as how to convert to rectangular coordinates and various benefits of this and that.

I wonder if they ever actually built one?

Thought you might like that.

Cheers,

Eric