sony

[Michael Hastings]

I'd check again Michael - it looks like Rot45 Bayer to me.

Graeme

Looks like it to me too.

All I've been saying all along is that look at the pixel counts! It all boils down to "if you double or triple the pixel count you'll get a better resolution image" - like wow. That's earthshattering!

Graeme

I agree - probably not earth shattering. It looks like sony is saying that for the same size sensor they are using twice as many (smaller) pixels to generate a 4K output and rotating them 45 degrees presumably because they feel it gives some additional benefit in readout processing or whatever.

Maybe they are trading off some sensitivity or Dynamic range for speed or improved colors or whatever (remains to be seen in the actual camera). I understand they pooh-poohed bayer in the past, and maybe they are admitting that bayer or bayer-like processing isn't so bad - good for Red, Canon, Nikon, etc - you all have every right to an "I told you so" moment.

All of these cameras are based on a mix of technologies. As parts of the mix develop, the engineering choices change. So I'm not sure what the big problem is here that people are so up in arms about.

 

[Graeme Nattress]

Sony have a graph somewhere in their slides showing DR falling off as pixel size decreases.

Graeme

 

[Andrew M.]

We were discussing this subject in the past but now maybe it is bit too much of information out of the door.

But from the past without going in to the details, Bayer patter lends itself to mathematical coincident of color when demosaicing. I am talking about XY coordinates.
It is giving us perfect position of color spots. Very good and useful by-product of Bayer pattern.

Converting upresing correcting is much more accurate. Graeme was explaining it in the past and I am not surprised that he doesn’t want to elaborate on this anymore. Sony in turn is good in RGB strips and has a whole library and past experience with it.
Finally they admitted that CMOS is way to go.

So let’s do CMOS the RGB way and since most screens are RGB and RGB striped it looks like logical choice. Sony has a lot of experience with it.

Well, for the TV signal and transmission, maybe, but for cinematography where you have heavy post, zooming, rotating, CCing, etc these RGB stripes and color not coincidental pixels may get in the way. Nyquist will be still alive here big time.

Maybe the offset of .000001 versus .0000015 looks very small but it is still 50% difference.

I think Sony needs some 2 years of playing with it and maybe they will figure it out.

 

[Bruce Allen]

All I've been saying all along is that look at the pixel counts! It all boils down to "if you double or triple the pixel count you'll get a better resolution image" - like wow. That's earthshattering!

If you double or triple the pixel count, you'll get a better resolution image.

And if you turn the bayer sensor 45 degrees and align the green pixels so that they are at a 1:1 relationship to the desired 4K output, you enable a neat hack that allows Sony to produce a realtime 444 4K image. In exchange for some quality loss vs a correctly-aligned 6K Bayer. But it's still better than a 4K Bayer (with unsophisticated decode).

For Sony:
4K Bayer with unsophisticated realtime deBayer = too low quality
4K Q67 (which looks like 6K Bayer at 45 degrees to us) = enough quality at realtime
6K Bayer with deBayer and scale = great quality but impossible to do realtime right now

Graeme, if you look at things this way (from a performance & "good enough" quality standpoint) not just a pure quality standpoint, it makes perfect sense and is a good compromise for them right now.

And for everyone who doesn't see this as a 4K BROADCAST solution... what do they hold up RIGHT AFTER showing their Q67 slides:
http://provideocoalition.com/index.php/awilt/story/hpatr2010_4/P1/

Oh hey, a HDCAM SR solid-state format... that stores compressed 4K 4:4:4 RGB.

Anyway, hope the Red guys are working on a RedRocket onboard module for Epic. The broadcast folks don't care if 6K Bayer is better if they can't get 4:4:4 4K out of the camera.

Bruce Allen
www.boacinema.com

 

[Graeme Nattress]

Why an unsophisticated decode? In such a camera memory bandwidth is tighter than absolute processing, so doubling the number of pixels coming is might make things worse, not easier.

As for broadcast - the bottleneck is the broadcast bit, not the camera bit. Broadcast, it would seem, cares not about 4:4:4 either.

Graeme

 

[Bruce Allen]

Why an unsophisticated decode? In such a camera memory bandwidth is tighter than absolute processing, so doubling the number of pixels coming is might make things worse, not easier.

Graeme, all through the thread, you compare Sony's Q67 4K sensor to a 6K Bayer sensor.

Now when it comes to realtime output, you swap your comparison to a 4K Bayer sensor.

As for broadcast - the bottleneck is the broadcast bit, not the camera bit. Broadcast, it would seem, cares not about 4:4:4 either.

Graeme

If the bottleneck is not the camera bit, please point me to a broadcast-style camera that can output 4K right now in realtime. Red One sure can't. Can Epic?

4:4:4 is less important - but did you read that Provideocoalition link? They've got a 4K 4:4:4 MPEG-4 compressed HDCAM SR recording system - so might as well send it a 4K 4:4:4 signal, right?

Bruce Allen
www.boacinema.com

 

[Sanjin Jukic]

And for everyone who doesn't see this as a 4K BROADCAST solution... what do they hold up RIGHT AFTER showing their Q67 slides:
http://provideocoalition.com/index.php/awilt/story/hpatr2010_4/P1/

Bruce,

those Sony 4K presentation slides were shawn here at the page 5 of this tread much earlier before Provideocoalition put them on their page...

Also I mostly agree what you have said about Sony 6K to 4K de-Bayer Q67 concept just because Sony want to improve 4K IQ (Image Quality)

giving much better start up for their demosaicking algorithm in the camera that can perform (can do better colorimety, color rendering, etc,...)

and that is actually the key to the visual quality of any digital camera.

 

[Graeme Nattress]

Bruce, my argument would be that you could just use a ~5k traditional Bayer, get a measured 4k and negligible aliasing instead of "6k as 4k" Rot45 Bayer. You'd be using less pixels to get the measured result desired. Or you could use the same number of pixels and get a better image (more resolution, less aliasing). Like a good demosaic would be a bottleneck when realtime MPEG4 (much more complex than a demosaic algorithm) on 4k data isn't?

As for realtime - JVC was demoing it a couple NABs back, as were NHK at 8k resolution.

Let's go back to what the main point is - the Sony slides that claim lower aliasing and greater resolution for their Q67 which are entirely to do with doubling the pixel count, rather than a change in orientation of the pixels by rotating them 45 degrees.

Graeme

 

[Bill Anderson]

The issue is being slowly eroded by personal ideas about how Sony's concept might fit a particular niche, or how it might work well for this and that... . And that is all very informative, but it does not deal with the Sony issue regarding Q67 over Bayer.

 

[Andrew M.]

And for everyone who doesn't see this as a 4K BROADCAST solution... what do they hold up RIGHT AFTER showing their Q67 slides:
Bruce Allen
www.boacinema.com

Bruce, I can't find 4K recording sollution by Sony.
Is the 4K verssion already out?

This is only 2K
http://pro.sony.com/bbsc/ssr/product-SRW5800/2/

But here they mentioned support up to 4K
http://nao.sonyelectronicscommunity.com/blog_post/?contentid=7878796312236267878

I think it is just support for 4K DPX file storage on the tape and I don’t think it is a real time.




Andrew

 

[Bruce Allen]

Like a good demosaic would be a bottleneck when realtime MPEG4 (much more complex than a demosaic algorithm) on 4k data isn't?

Not a fair comparison - realtime MPEG4 is something Sony presumably is developing dedicated chips for.

That development cost can be spread across multiple departments and that chip can go in many different broadcast products.

Also, that chip could live on an outboard recorder that is fed the 444 4K signal from the camera. Then you can keep the camera head itself really simple.

This is completely different from the camera dept getting a custom high-quality deBayer chip manufactured for them, which will be specific to their camera and would sit on the camera head itself, creating heat and requiring much power.

Think of a show like a Formula 1 Grand Prix with one of those little remote heads flying over the pits... Sony (and the broadcasters) want a nice 4:4:4 4K signal coming from them, with minmal fuss. They don't want to be sent proprietary RAW because probably they're intercutting it with Panasonic 4K cameras and JVC cameras too.

4K Bayer they deem too low quality. A 5K - 6K Bayer chip producing a 5-6K image that is then scaled down requires a bit too much processing power and memory for their current chips. A 4K Q67 chip seems to be nicely in the middle for them.

Bruce, I can't find 4K recording sollution by Sony.
Is the 4K verssion already out?

Presumably it'll come out the same time as this camera. They announced it Feb 2010. Pic attached from Adam Wilt's Provideocoalition article.

Bruce Allen
www.boacinema.com

 

[Graeme Nattress]

"This is completely different from the camera dept getting a custom high-quality deBayer chip manufactured for them, which will be specific to their camera and would sit on the camera head itself, creating heat and requiring much power." - and that's compared to a completely new algorithm in silicon for a new sensor arrangement when they can just borrow the IP from their DSLRs that already do quality demosaic in hardware?

Bruce, don't get me wrong. I follow your argument, but I also have very good knowledge of what can be done in hardware at these resolutions and frame rates, and I also know how I could produce a better image with either fewer or the same number of pixels, and how to keep the amount of processing way down, but I'm not likely to give that level of information away on a public forum.

Graeme

 

[Ian Laurie]

I am starting to get a head ache. I will stand up and say this right now...I have no idea how most of this stuff works. But I do know logic. Logic tells me that a Bayer pattern, turned on a 45, is still a Bayer pattern. It still requires debayering. I am not sure why this seems to be more efficient. but lets say it is, why not just have software look at the image diagonally instead of turning the sensor? It also seems to me that if you were to have the equal number of pixels on the sensor and you still use the weird overlapping grid they show in THEIR slide, you would actually have blocks made up of green/blue and then a separate block of green/red.

Simple math and logic dictates that by halving the size of the pixels, and doubling the number in equal amount will make no change whatsoever in the percentage of return. Neither will turning it on a 45 degree angle. you still have 25% blue pixels, 25% red pixels and 50% green pixels.

Diagonal or horizontal, you still would have the same mathematical percentage of retained info. So....

Either Sony is really really bad at math (which i highly doubt and would defend their engineers credibility to the end cause i love their products) or this is marketing BS. And for the price that theoretical camera will run ( no doubt at least 100k, i think a fair price to attribute to it considering past professional camera systems) what would stop them from putting in a rocket style card in camera. Surely RED could do it if they wanted to bump the price up.

When it comes down to it, I just don't see the math or the logic. I can see a camera with no greater capabilities, except for those that are justified by a higher price tag, which is fine for those who need those capabilities built in. What I don't see is how it is any easier to deal with.

out of curiosity, would someone with a far greater mind let me know why a true diamond pattern hasn't been done? just thinking it would be interesting to see what the benefits would be to doing something like a diagonal stripe using vertical diamonds (squares elongated at 2 points)

 

[Eric Lange]

Just to push the imagination a bit. (sorry I just realized Ian you just posted the same thing, I guess great minds think alike?)

I don’t understand where the idea of a bandwidth limited inferior real time de-moisaic comes from?

They mentioned several times that they had PS3 GPU(s) on board. Could you not have conventional types of de-bayering algorithms applied to the stored luminance values, but running with groups of photo sites on various strips running along their natural 45 degree boundaries? It could work as a totally conventional de-bayer (just running spatially at 45 degrees)? Then the temporarily stored values are re-sampled using an inverse-mapping algorithm to in effect render out the rectilinear set of color values that could be further compressed or coded any way you like? The inverse mapping technique is no different than what is employed in graphics cards for conventional rendering and in particular texture mapping. The advantage being is that high precision sub pixel values can be used for sampling and this can be carried out lightning fast given that the architecture has to be organized in a highly parallel fashion given that high res CCD(s) are normally as slow as you can possibly get.

In spite of amazing things that can be imagined, it still doesn’t get round the idea that you don’t get 100% green spatial channel from a device that is only covered 50% on its surface with green photo sites.

The problem here is with perceived value and perceived technical capability.

They say Bayer (with such disdain, say “ Baaaaaaaaaaaayurr”, you have to stick out your chin on the “yurr”) is only 33%

But Q 67 is 67% with a 100% green (this sounds like a soap powder commercial).

This is designed to mislead one to believe that Q67 (soap powder) is twice as good as old fashioned “Baaaaaaaaaaaayurr”.

Still nobody has explained where this mysterious 100% green channel comes from?

 

[Ian Laurie]

Thanks Eric, What i was trying to say and ask, but with clearly much more engineering intelligence.

 

[Ian Laurie]

Ha, My mom would be so proud having me called a great mind.

 

[Daniel Browning]

4K Bayer with unsophisticated realtime deBayer = too low quality
4K Q67 (which looks like 6K Bayer at 45 degrees to us) = enough quality at realtime
6K Bayer with deBayer and scale = great quality but impossible to do realtime right now

But you can do the *exact* same processing with Bayer as you can with Q67, so there is no such advantage.

• Q67: Read 13 pixels in an "X" shape for each output pixel
• Bayer: Read 13 pixels in a "+" shape for each output pixel

Specifically, to calculate the Q67 output pixel value, read a certain 12 of the surrounding pixels in an "X" shape (given from top to bottom, left to right, which coordinates relative to 45-degree index):

[-2, 0], [-1, -1], [0, -2],
[-1, 0], [0, -1],
[-1, 1], [0, 0], [1, 1],
[0, 1], [1, 0],
[0, 2], [1, 1], [2, 0],

In Bayer, you read the exact same 12 surrounding pixels, but in a "plus sign" shape instead of an "X" shape. Again, top-to-bottom, left-to-right:

[0, -2]
[-1, -1], [0, -1], [1, -1],
[-2, 0], [-1, 0], [0, 0], [1, 0], [2, 0],
[-1, 1], [0, 1], [1, 1],
[0, 2]

Here's the kicker: both are going to give you the *exact* same pixel value. It doesn't matter whether you rotate the pixel 45 degrees or not -- either way you're just reading 13 pixels to come up with 1. You can do that just as easily with normal Bayer as you can with Q67.

 

[Bruce Allen]

But you can do the *exact* same processing with Bayer as you can with Q67, so there is no such advantage.

• Q67: Read 13 pixels in an "X" shape for each output pixel
• Bayer: Read 13 pixels in a "+" shape for each output pixel

1. Your description is not correct. With Bayer you are outputting one pixel for every sensor site (be it red, green or blue). With Q67, you are outputting one pixel for every green sensor site. There is a green sensor for every pixel.

If you do a Q67 decode of the same megapixel sensor, you end up with less pixels than with a Bayer decode.

Still nobody has explained where this mysterious 100% green channel comes from?

See above. THERE IS A GREEN SENSOR FOR EVERY PIXEL output. Unlike Bayer.

2. For Q67, they're not reading 13 sensor sites. They're reading 5. Eg the green sensor site at the pixel, the 2 diagonally adjacent red pixels and the 2 diagonally adjacent blue pixels.

Bruce Allen
www.boacinema.com

 

[Bruce Allen]

Bruce, don't get me wrong. I follow your argument, but I also have very good knowledge of what can be done in hardware at these resolutions and frame rates, and I also know how I could produce a better image with either fewer or the same number of pixels, and how to keep the amount of processing way down, but I'm not likely to give that level of information away on a public forum.

OK awesome Please go forth and kick their ass. If RED can release a realtime 4K broadcast camera before Sony, it'd be great. Seriously. Show that they were wrong to play it safe with the Q67 compromise.

But that's very different from saying that their design makes no sense at all and is purely a marketing ploy.

You guys push for quality and value with as few compromises as possible - which to the uninformed viewer (eg me also sometimes seems to cause you guys some schedule problems. Sony has made a compromise design with Q67 that to the uninformed viewer seems to be on schedule. Eg:
- slightly better color resolution than the current top digital cinema cameras incl Red One
- realtime 4K output
- likely on schedule - eg not dependent on a fancy ASIC that might run late

Later on they can replace it with a 5K or 6K Bayer sensor downsampled and charge more.

So yeah, please, kick their ass by showing it's possible to beat them at their own game without compromise from the beginning. It'd benefit us filmmakers for sure.

My only argument is with the theme on this thread that Q67 is primarily some kind of deranged marketing attack. I see it as an engineering-driven compromise that seems on track to result in a shipping product which might be able to do things no other shipping product can do (eg 4K realtime output at higher color resolution than Red One).

Bruce Allen
www.boacinema.com

 

[Ian Laurie]

1. Your description of Bayer is not correct. With Bayer you are outputting one pixel for every sensor site (be it red, green or blue). With Q67, you are outputting one pixel for every green sensor site. There is a green sensor for every pixel.

If you do a Q67 decode, you end up with less pixels than with a Bayer decode.

2. For Q67, they're not reading 13 sensor sites. They're reading 5. Eg the green sensor site at the pixel, the 2 diagonally adjacent red pixels and the 2 diagonally adjacent blue pixels.

Bruce Allen
www.boacinema.com

so why not just change the bayer decode to target only green sensor sites. the logic is still flawed. 45 degrees does not change this one bit.