sony

[Sanjin Jukic]

Just to push the imagination a bit. (sorry I just realized Ian you just posted the same thing, I guess great minds think alike?)

I don’t understand where the idea of a bandwidth limited inferior real time de-moisaic comes from?

They mentioned several times that they had PS3 GPU(s) on board. Could you not have conventional types of de-bayering algorithms applied to the stored luminance values, but running with groups of photo sites on various strips running along their natural 45 degree boundaries? It could work as a totally conventional de-bayer (just running spatially at 45 degrees)? Then the temporarily stored values are re-sampled using an inverse-mapping algorithm to in effect render out the rectilinear set of color values that could be further compressed or coded any way you like? The inverse mapping technique is no different than what is employed in graphics cards for conventional rendering and in particular texture mapping. The advantage being is that high precision sub pixel values can be used for sampling and this can be carried out lightning fast given that the architecture has to be organized in a highly parallel fashion given that high res CCD(s) are normally as slow as you can possibly get.

In spite of amazing things that can be imagined, it still doesn’t get round the idea that you don’t get 100% green spatial channel from a device that is only covered 50% on its surface with green photo sites.

The problem here is with perceived value and perceived technical capability.

They say Bayer (with such disdain, say “ Baaaaaaaaaaaayurr”, you have to stick out your chin on the “yurr”) is only 33%

But Q 67 is 67% with a 100% green (this sounds like a soap powder commercial).

This is designed to mislead one to believe that Q67 (soap powder) is twice as good as old fashioned “Baaaaaaaaaaaayurr”.

Still nobody has explained where this mysterious 100% green channel comes from?

Above is illustrated Sony "soap powder" that maybe can work but anyway you must pay probably about

the same price as a F35 and also possible that could again get a "fat duck" camera as it is F35.

Fixing rolling shutter is a good thing...

But at the and of the story all that lead to conclusion: Advantage RED EPIC.

Just because of lightweight ergonomics, easy of use and price...

 

[Bruce Allen]

Above is illustrated Sony "soap powder" that maybe can work but anyway you must pay probably about

the same price as F35 and also could again get a "fat duck" camera as it is F35.

Fixing rolling shutter is a good thing...

But at the and of the story all that lead to conclusion: Advantage RED EPIC.

Just because of lightweight ergonomics, easy of use and price...

Sanjin, I am with you 100% on this

And "fat duck" is the most awesome F35 description ever.

Bruce Allen
www.boacinema.com

 

[Daniel Browning]

1. Your description of Bayer is not correct. With Bayer you are outputting one pixel for every sensor site (be it red, green or blue).

Is there some physical law or government restriction that forces you to do that? Of course not. Just as there is nothing that forces Sony to decode their 6K sensor at 4K. They only do it to hit real-time output at lower power levels. You can do the exact same thing with unrotated Bayer.

Of course, you don't normally throw away so much resolution like that, but if you can't figure out a smarter way to handle 6K of data (such as REDCODE), I guess tossing out resolution is your only alternative. But again, you can throw away resolution in normal Bayer just as easily as unrotated Bayer.

See above. THERE IS A GREEN SENSOR FOR EVERY PIXEL output. Unlike Bayer.

If you live in the Former Soviet Republic of Panavsonystan, then you're right. Chairman Mao dictated that you are only allowed to demosaic Bayer at less than full resolutions if the pixel is rotated 45 degrees, so you can't do that without going against the Party. But over here in the free world, we allow ourselves to demosaic pixels at whatever resolution we need, rotated or not.

2. For Q67, they're not reading 13 sensor sites. They're reading 5. Eg the green sensor site at the pixel, the 2 diagonally adjacent red pixels and the 2 diagonally adjacent blue pixels.

It doesn't matter how many they are reading or what pattern they read them in -- you can always do the exact same thing in unrotated Bayer that you can with rotated Bayer.

 

[Graeme Nattress]

Bruce, indeed all engineering is the art of the compromise.

"OK awesome Please go forth and kick their ass" - oh yes indeed. That's always the plan.

Graeme

 

[Eric Lange]

Even the crudest de-Bayer will use four neighboring green photo sites for a single pixel, so by SONY’s marketing logic I have 4 greens contributing to every “pixel” as opposed to one, so I have 400% more than Q67!

By the same logic I could call my product Q600 because I have two greens for every blue and red.

From the diagrams, they are saying that in totality, their green photo-sites add up to the same as a 100% green area sensor coverage, being just the same as a green channel from a true RGB system pictured above. It is respectfully submitted, that this is false, and misleading.

No matter how you sample and resample, the green photo sites can only physically pick up 50% of the corresponding spatial signal and area unless this is a multilayered device or there are bi-modal filters on the blue and the red photo sites, and even then it’s a real fiddle.

...I hope it’s understood that the green diamond only accounts for 50% of the total area defined by the smallest cell marked with the black lines. I.e a 45 degree rotated square that fits into a rectilinear square, said rotated square only accounts for 50% of the area of the larger rectilinear square; right?

 

[Bruce Allen]

Bruce, indeed all engineering is the art of the compromise.

"OK awesome Please go forth and kick their ass" - oh yes indeed. That's always the plan.

Graeme

All of Reduser is behind you. Seriously, how awesome would it be if Sony got all of the broadcasters onto the 4K bandwagon... and then everyone said "OK!", bought a 4K vision mixer from Sony... but then went and bought a ton of RED cameras instead because they plugged right in to it and delivered better quality for less money?

If you live in the Former Soviet Republic of Panavsonystan, then you're right. Chairman Mao dictated that you are only allowed to demosaic Bayer at less than full resolutions if the pixel is rotated 45 degrees, so you can't do that without going against the Party. But over here in the free world, we allow ourselves to demosaic pixels at whatever resolution we need, rotated or not.

It doesn't matter how many they are reading or what pattern they read them in -- you can always do the exact same thing in unrotated Bayer that you can with rotated Bayer.

This is hilarious and awesome - but please look at your own diagram. If you choose to demosaic a 6K Bayer as 4K using the Q67 method of "one green sensor per one pixel" then you have a problem because the green sensors are not vertically aligned line-for-line. They are diagonally offset from each other - eg by 1/2 a pixel for every odd line. Which presents a problem since 4K displays likely won't be laid out like that.

Bruce Allen
www.boacinema.com

 

[Ian Laurie]

This is hilarious and awesome - but please look at your own diagram. If you choose to demosaic a 6K Bayer as 4K using the Q67 method of "one green sensor per one pixel" then you have a problem because the green sensors are not vertically aligned line-for-line. They are diagonally offset from each other - eg by 1/2 a pixel for every odd line. Which presents a problem since 4K displays likely won't be laid out like that.

Bruce Allen
www.boacinema.com

So rotate the debayer algorithm rather than the sensor. It is the same thing. seriously, look at the normal bayer pattern, tilt your head 45 degrees, and voila! the green photosites align. I stand by the notion that no matter what angle you put the sensor at it is still a bayer sensor, and still captures colour information at the same 50%G 25%B 25%R. everything else is programming choices.

 

[Andrew M.]

So rotate the debayer algorithm rather than the sensor. It is the same thing. seriously, look at the normal bayer pattern, tilt your head 45 degrees, and voila! the green photosites align. I stand by the notion that no matter what angle you put the sensor at it is still a bayer sensor, and still captures colour information at the same 50%G 25%B 25%R. everything else is programming choices.

Not quite, the beauty of Bayer patter is that no matter what 3 adjacent photosides you pick and no matter at what XY coordinates on the sensor, you always have RGB in it.

On Q67 sensor you have to skip some pixels to get adjacent RGB trios, leaving the redundant pixels behind. Bayer is 100% tightly covered with RGBs trios without any pixels skipping.

 

[Bruce Allen]

So rotate the debayer algorithm rather than the sensor. It is the same thing. seriously, look at the normal bayer pattern, tilt your head 45 degrees, and voila! the green photosites align.

Okay. I'm with you. Now I want to watch this on an HD display that is not mounted diagonally. So I need to rotate the image 45 degrees at the end.

You're saying that there is NO penalty (performance or image-quality-wise) from performing a 45 degree rotation on your image after it's been debayered? Really?

Read this article:
http://www.fxguide.com/featured/Keeping_Your_Renders_Clean/

There is a significant softening involved each time you rotate an image.

Bruce Allen
www.boacinema.com

 

[Andrew M.]

Anyway, why to teach competition how to do it right?

The best about the Bayer pattern and it demosaicing was not even mentioned here.
What is the most important here is the fact that Sony confirms that 4K is the future standard that they want to support.
That 4K is the future standard not only for cinema cameras but its sister UHDTV.

This thread should be closed, why to give Sony any ideas. I invested my time and affords in RED technology so I don’t want to dilute my investments

 

[Stuart English]

What is the most important here is the fact that Sony confirms that 4K is the future standard that they want to support.

Agreed. Sony is actively promoting a 4K production universe (camera to be observed at some future date) and RED is shipping its 5K Bayer RAW (true 4K RGB delivered resolution) camera, and is about to commence shipments of it's 4K true RGB resolution RED RAY media player.

All good.

 

[Eric Lange]

.

If you live in the Former Soviet Republic of Panavsonystan, then you're right. ...

I thought that was pretty funny.

 

[Ian Laurie]

Okay. I'm with you. Now I want to watch this on an HD display that is not mounted diagonally. So I need to rotate the image 45 degrees at the end.[/url]

No, you need to rotate the debayer algorithm, not the final out put.

You're saying that there is NO penalty (performance or image-quality-wise) from performing a 45 degree rotation on your image after it's been debayered? Really?[/url]

Nope never said that. you did.

There is a significant softening involved each time you rotate an image.

Bruce Allen
www.boacinema.com

Yes there is. but i don't see why turning the sensor 45 degrees and changing the algorithm to suit that is any more brilliant than leaving the sensor the way it is and changing the way the algorithm reads the sensor. sorry if that was to hard to understand.

 

[Ian Laurie]

This really does sound like the dumbest thing to me. It really seems like a whole lot of non sense. when it comes right down to it, they took a square bucket that has 3 compartments, rotated it then said it captures more colours. it doesn't add up. period.

If the advantage lies in the idea that by reading the colour information a different way by putting it 45 degrees, just write the debayering software to calculate the colour at 45 degrees rather than rotate the sensor. It just sounds incredibly stupid. 45 degrees or not, it is still capturing the same colour information, its what happens after that point that is going to change it.

 

[Bruce Allen]

Agreed. Sony is actively promoting a 4K production universe (camera to be observed at some future date) and RED is shipping its 5K Bayer RAW (true 4K RGB delivered resolution) camera, and is about to commence shipments of it's 4K true RGB resolution RED RAY media player.

All good.

100% agree! All is indeed very good in the REDverse!

Gotta get off this thread now and get back to the business of making content.

I will make one last attempt to try to explain Q67 to Ian then I am gone from this thread:

Yes there is. but i don't see why turning the sensor 45 degrees and changing the algorithm to suit that is any more brilliant than leaving the sensor the way it is and changing the way the algorithm reads the sensor. sorry if that was to hard to understand.

OK. Please try to think this through. Think about this algorithm you're proposing: whereby a 6K Bayer sensor is "read diagonally" in a Q67-like way - and you somehow end up with a pristine 4K image with minimal computation in one step.

What you do when you hit pixels where the green sensor doesn't actually line up with the pixel being output, due to the mismatch between the diagonal pattern of green sensors in a regularly-oriented Bayer sensor?

It doen't matter whether your algorithm works horizontally, vertically or diagonally. If you're using a non-rotated Bayer sensor of the same size as the Q67 sensor, half of your green sensors don't map 1:1 to the pixels being output, period. In the end you're going to have to do some interpolation between adjacent green values. Which is bad.

You can play word games that make it sound like working horizontally across a diagonal sensor is the same as working diagonally across a horizontal sensor - but you are missing the fact that a Q67 sensor (or Bayer turned 45 degrees) has a perfect grid of green sensor elements that align 1:1 to the pixels being output if you are decoding it via the Q67 method.

By contrast, a horizontal Bayer sensor has green sensor elements which do not line up with the pixels being output if you are decoding it with the Q67 method. Every second line the green sensors are "in between" two output pixels. This will result in an image that is by nature softer.

Only you are proposing that it makes sense to decode a Bayer sensor diagonally in a Q67-style way though. The true 4K Q67-beater is a 5K or 6K Bayer, sampled down nicely (which takes more computational work).

I'm done repeating myself. Either you bother to read what people say - and can therefore should have got Q67 by now - or you don't. It looks like a lot of people don't.

That's fine by me - I am in the RED camp so if it turns out that their competition is doing something that is utterly incomprehensible to the average filmmaker, all the more power to RED.

Bruce Allen
www.boacinema.com

 

[Eric Lange]

Maybe this explains the 100% green channel response…

(Lord Percy discovers 100% pure green)

www.youtube.com/watch?v=TkZFuKHXa7w

 

[Eric Lange]

Bruce maybe this might help...

The alignment of photo sites dosen’t matter considerably in terms of the rectilinear array of pixels that you have to output. In CG and may other simple image processing operations, you first take an on screen pixel, and then you decide from what parts of another image, in this case the captured Bayer pattern image that you want to draw values from. You can use any number of sampling points that you wish, carry out any types of computation you wish in any combination or pattern that you want. You can apply all sorts of non-linear weighted functions. You can designate any number of xy positioned sub pixel point samples (floating point, single precision or double precision) to sample from and do math with. You can even carry out least squares adjustment computations from redundant samples to come up with a more probable spatial and radiometric result. The shape of the array of sub pixel precision points used to sample the Bayer image can be in the shape of Crusty the Clown if you want; this doesn’t affect the ability to output a rectilinear array of pixels. Then you can turn that on screen pixel to the computed and desired color value. Then you move onto screen pixel [1] from [0] and do the same.. and move onto the next and so on. Its just rendered, its no big deal really.

 

[Bruce Allen]

Bruce maybe this might help...

The alignment of photo sites dosen’t matter considerably in terms of the rectilinear array of pixels that you have to output. In CG and may other simple image processing operations, you first take an on screen pixel, and then you decide from what parts of another image, in this case the captured Bayer pattern image that you want to draw values from. You can use any number of sampling points that you wish, carry out any types of computation you wish in any combination or pattern that you want. You can apply all sorts of non-linear weighted functions. You can designate any number of xy positioned sub pixel point samples (floating point, single precision or double precision) to sample from and do math with. You can even carry out least squares adjustment computations from redundant samples to come up with a more probable spatial and radiometric result. The shape of the array of sub pixel precision points used to sample the Bayer image can be in the shape of Crusty the Clown if you want; this doesn’t affect the ability to output a rectilinear array of pixels. Then you can turn that on screen pixel to the computed and desired color value. Then you move onto screen pixel [1] from [0] and do the same.. and move onto the next and so on. Its just rendered, its no big deal really.

Eric, you are talking about theory. I am talking about something that can be achieved realtime without custom chips. With as little power draw as possible. EG where processing power is so limited that they can't even do a full traditional debayer at 6K.

Again, if you look at that fxguide.com link you'll see that even the best algorithms still cause softness with just a 45 degree rotation.

So - you are telling me that there is no benefit to aligning the green sensor elements with the actual pixels being output? VS having the green sensor elements at a 45 degree angle so that every second line they are not aligned with the pixels and you have to average between them? Even for a realtime device?

Anyway, I'm out. The only people left arguing with me aren't really bothering to think things through.

Bruce Allen
www.boacinema.com

 

[Eric Lange]

In the next 18 months these things will not seem so theoretical even to you. The raw horse power of generalized GPU architectures are only just being realized; that’s the beauty, custom hardware is not so important any more for serious number crunching and signal processing.

[Just as an example I can run frame sequential stereo rendering going at 60 HZ on two HD monitors (at the same time both running stereo) , this is effectively the same processing area as 4k mono (because of the double frame rate), and this is rendering extremely complex 3d data where the number of per pixel (sub pixel accuracy) operations far exceeds what is required for a frugal de-Bayer. With generalized GPU architecture, access to more direct programming such a CUDA allow for true high performance computing in a truly multi parallel way on relatively standard and inexpensive hardware. To me the fact that SONY wants to stuff GPUs from the PS3s in their cameras and “video decks” is not at all surprising to me. These GPUs are extremely powerful little beasties that can be linked together into parallel nodes to make quite a monster!]


Thanks for having a go anyway.

Cheers

Eric

 

[David Rasberry]

I can see how Sony might read out "Q67" from a higher density higher pixel count array like they read a striped array and get one RGB pixel 2:1:1 sampled from a 4 pixel block with simpler processing than interpolating that same 4 pixel block to produce 4 RGB pixels. But the Sony diagram is FUD. It indicates that you can read out half of a photo site to get that RGB pixel which is physically impossible. It would also seem likely that you would get more aliasing issues with that approach, not less.