Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

General Demosaicing Questions

Graeme,

One thing that confuses me about the Q67 is that, as I understand it, one of the strengths of the Bayer pattern in its normal orientation is that we get a continuous line of green photosites along a diagonal line; which helps greatly avoiding aliasing with our common-world horizontal and vertical lines we see around us. Wouldn't rotating the CFA orientation 45-degrees negate this benefit, making the continuous green photosite lines to now to run parallel with horizontal and vertical lines in our environment and possibly interrupting vertical lines or falling half on, half off?
 
Christopher,

I know you addressed this ? to Graeme, but I believe I have an answer for you.

Your concern would be the case if the image were displayed with 45 degree rotated pixels. But that's not the case. Q67 orientation is for image signal acquisition only. The image signal is still mapped to a normally oriented 4K grid.

So you should not see more aliasing due to the jagged vertical and horizontal nature of the Q67 sensor, b/c you're not playing back on a display with its pixels aligned in that manner.

I believe that's correct, but I would love to hear Graeme's thoughts ;).
 
There's more at play than just co-incidence of pixel orientation on the sensor to the display pixels. I get your meaning, but I'm talking about the effective diagonal resolution benefit of the continuous green pixels in the array, which creates better horizontal and vertical resolution (oddly enough, when the pixels are diagonally continue, but it's true)... Rotating the array 45 degrees on the Q67 now aligns the green pixels horizontally and vertically, now giving the effective resolution benefit in the diagonal direction, which my point was less of a common real-world alignment with architecture, horizon ect...

Just was curious to hear Graeme's take on that since this has much to do with how you demosaic the Bayer information, of which the Q67 still is, but I know they are handling that info in a different manner.
 
Well, a simple experiment would be to take a traditional Bayer pattern sensor and rotate the camera 45 degrees and see if there's less or more aliasing on horizontals and verticals. Indeed that's why we measure on circular zone plates as then you see aliasing in all directions.

When Sony show their mocked up images to show less aliasing on 45° Bayer, remember though they've doubled the pixel count. It's that doubling that lowers aliasing. Then they show "Q67" after some algorithmic chroma moire reduction, but "forget" that such algorithms apply equally to traditional Bayer and indeed, we've had one in place for use, should it be necessary, since the earliest of days.

If we do a proper comparison where we keep the pixel count the same, when you turn a pixel on it's diagonal you've now increased the horizontal distance between pixels by a factor of root(2) (assuming square pixels). Also, you're no longer approximating a box sample in that direction, but you've got a different sampling structure which could also cause issues.

Graeme
 
Christopher and Graeme,

I think there is an essential aspect of Q67 that's being skimmed over. The whole point of Q67 is that there is no demosaicing. The alignment of the smaller pixels to the to 4K grid effectively does a demosaic. So no demosaicing algorithm is applied.

While it's true that Q67 is a Bayer pattern rotated 45 degrees, it's important to realize that unlike the Red, Q67 does not have a one-to-one relationship of photosites to pixels. Instead, each square pixel is made up of one centrally located diamond shaped green photosite. The corners of the square pixel are comprised of quarter samples of the red and blue photosites that surround the green photosite. So samples from five photosites are blended to create a color value for the pixel. This is a defacto, somewhat crude, physical type of demosaicing. But it's not interpolative, like the Red and other Bayer sensor demosaicing algorithms are.

As for V/H vs diagonal resolution, Q67 should have slightly less resolution along the diagonal, not greater. That's b/c on the V and H axises, the green pixels are contiguous by virtue of their corners touching. But along the diagonal axises, the green pixels are not touching and are separated by red or blue pixels. This should produce chroma aliasing at the finer resolutions that will first show up along the diagonals.

But Graeme's main point of the value of good interpolation and demosaicing algorithms I'm sure is true. It's entirely possible that Q67 is solving a problem, the difficulty of demosaicing, that really has already been surmounted. And to get around demosaicing, Q67 has to use smaller pixels, which introduces problems of noise and sensitivity.

Also, while Q67 doesn't interpolate per-se, it still has to "image processes" to remove chroma aliasing (that I described above).

I hope that's accurate and helpful.
 
Even if they don't process on the greens, they still need to process on the red and blue pixels. Higher noise and lower sensitivity (as diagrammed by Sony) would seem to be an expensive price to pay for slightly simpler demosaicing to me. Not least as that for the same number of pixels you'll end up with more moire and more aliasing, not less. I think I'll stick to demosaicing myself...

Graeme
 
if you actually watched sony's presentation and still think they're going to give you what you really want, i don't know what to tell you. i'm tired of hearing anything about sony and what they are doing. next person that mentions the word 'tape' gets it.
 
Graeme Nattress said:
Even if they don't process on the greens, they still need to process on the red and blue pixels. ...
Click to expand...

Graeme, I'm not sure what you mean by that? Each of the 4K pixels is constructed exactly the same way. It's true that the red and green photosites are split across pixels, but that's minimal processing.

But, FWIW, I do agree with your position that in 2011 debayering should not be seen as this unsurmountable task. And the price Q67 pays to avoid it seems awfully high. Also, as you said, applying a debayering algorithm to Q67 would add even more resolution.
 
To properly reconstruct the red/blue is non-trivial processing. I spend as much time worrying about reconstructing the red/blue, if not more actually, than working on the greens when I demosaic. A simple interpolation can be used, but it's very prone to artifacts and chroma moire. If you're going to the bother of properly working on the red/blue you've done enough processing to get to the point where you may as well be doing the greens also!

Graeme
 
Graeme,

Right, but you're doing that where you have a one-to-one relationship of photosites to pixels. So determining the R, B (or G) value of any pixel is non-trivial.

But with Q67, there is no interpolation b/c each 4K pixel aligns over one full photosite and the corners of four other photosites. So constructing the R, G and B values of a pixel seems to be much easier.

But, again FWIW, I do agree with you that it's hard to argue that debayering and interpolation are things that should be avoided. And using smaller pixels very well may be a solution that's worse than the "problem."
 
It doesn't make R B reconstruction easier though. That was my point. A simple bilinear interpolation will look just as bad as how things look when you do a simple bilinear on a traditional bayer. As Sony point out they says they use more advanced R B processing to remove chroma moire, so we can assume they're not trivially processing the R B.

Graeme
 
Okay, I see what you're saying. Thanks much Graeme.


P.S. So it's entirely likely that Sony is performing what's very close to debayering but calling it "image processing."

BTW, I think Hugo was being disingenuous when he made it sound like the Red does not do any sensor noise cleaning in camera. I may be wrong, but I have to think the Red indeed does this. Isn't that what's black balancing is for? To get a signature of the sensor's noise and use that to clean sensor noise from the sensor data? And I have to think that's not stored as metadata but is actually performed in camera. (Of course I could be wrong.)
 
:-) But do Sony?

Graeme
 
Peter Moretti said:
Graeme,

Right, but you're doing that where you have a one-to-one relationship of photosites to pixels. So determining the R, B (or G) value of any pixel is non-trivial.

But with Q67, there is no interpolation b/c each 4K pixel aligns over one full photosite and the corners of four other photosites. So constructing the R, G and B values of a pixel seems to be much easier.

But, again FWIW, I do agree with you that it's hard to argue that debayering and interpolation are things that should be avoided. And using smaller pixels very well may be a solution that's worse than the "problem."
Click to expand...

You can't read half a photo site. A photosite has to included or excluded from the RGB pixel both spatially and for value regardless of how it is processed.
 
David Rasberry said:
You can't read half a photo site. A photosite has to included or excluded from the RGB pixel both spatially and for value regardless of how it is processed.
Click to expand...

David,

Come, on. I don't literally mean reading only 1/4 of the photosite. I mean that the photosite is positioned so that it spans across the corners of four pixels.
 
:-) We know what you mean Peter. I guess the next question is how such an arrangement performs in practise, rather than Sony's simulations.

Graeme
 
Graeme Nattress said:
:-) We know what you mean Peter. I guess the next question is how such an arrangement performs in practise, rather than Sony's simulations.

Graeme
Click to expand...


The other SONY thread got sent to the Siberian Gulag of “off topic” did thrash a lot of this out, to the point where I (at least) was guilty of carpet bombing an equine graveyard or two.


I think Graeme is being super correct and super generous here.

It seems that there is a whole smorgasbord of very creative methods for more advanced “de-bayer”. It seems that there are many multiple sets and approaches that can be employed at every turn to skin this particular cat. I think that’s the nice thing about RED and the “Graeme thing” is that under the hood (perhaps) that nothing is absolutely set in stone, and perhaps it seems that there are many more possibilities and options for the future. In reality what SONY set out may not be that fundamentally different from other less well known schemes. I think more of what they are doing is about the physical chip with a zig-zag read out, that might provide fast and different types of processing (that also have their particular associated design tradeoffs).


Fundamentally looking at things for a moment with a Physisict’s eye, a bit like Graeme, he has made the point on numerous occasions that there IS NO FREE LUNCH! What SONY has brought to bear I believe is really THEIR IDEA of how the mix of various design trade-offs might play out to fit THEIR VISION of the types of application areas and markets they seek to serve. However RED has a different vision from SONY and hence their mix of design solutions and tradeoffs has produced a completely different type of “instrument”, way in advance. I used to think that all of the major companies in the cinematic market had too much competitive overlap all jostling for a tiny space. Nowadays I don’t think that any more. I would say that of the bigger ‘Outfits” the ones that employ a rather “Germanic” absolutely uncompromising design rationale, will in the long run become extinct Dinosaurs, (great amazing beasts as they are but not flexible enough to survive in a more dynamic environment).


So does anybody know what the realistic time frame for the SONY 4k offering will be? I.e. where you can actually shoot a real production with the thing? As I said before it’s all in the implementation, and I suspect a lot film makers are NOT going to like the look of a global shutter, this look is EVEN FURTHER away from this notional “ideal” of the “film/movie look”.


I kinda sense that “film purists” that are forced to eventually go digital will jump on the RED bus and not the SONY bus, purely because of the more sterile look from a globally shuttered system. I think Jim made that point a couple of years ago at least.
 
Eric,

Just as a fast read-out rolling shutter is a non-issue, so is a global shutter. Every CCD camera out there uses a global shutter, and no one complains about it. No one notices a slight tilt, or the lack of one, during a fast pan.
 
Been deep in code. Just coming up for air.

Graeme
 
You must log in or register to reply here.
Back
Top