"The Truth About 2K...Future of Pixels" via Creative Cow

[Robert Sloan]

**DISCLAIMER**
I am not associated with Creative Cow or Panavision.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

I'm not sure how many of you have seen this, but here's an interesting opinion piece on John Galt's vision of the future of digital cinematography. He's the SVP of Advanced Digital Imaging at Panavision, so there's some serious bias, I think.

An intriguing read to get some other's opinions, though:
http://magazine.creativecow.net/article/the-truth-about-2k-4k-the-future-of-pixels

Here's an excerpt:

So 4K is not these 8 mega pixel or 9 mega pixel or 10 mega pixel CMOS images for the Bayer pattern where they add up all the pixels in a row and say hey, we got 4K. The great perpetrators of that mythology have been RED and Dalsa. That's why I call these “marketing pixels." It's intentional obfuscation. Because they really do nothing to improve image quality. They may improve sales volume. But they don't do anything to quality.

Like I said, opinion. I think the slew of well-regarded DPs, Directors, and Producers that have come to use the RED cameras on their projects would argue with the last bits of that quote. Back-to-film resolution that beats that "DI" resolution, latitude that exceeds what film is capable of, and extreme low light performance are things I would consider "improvements to quality."

Anyone else have thoughts?

 

[David Mullen ASC]

We already discussed that article a while back.

If anyone wants to know how much each manufacturer's camera resolves per color channel, and how much aliasing they allow, all you have to do is point them at some charts. Nobody can hide anything ultimately, marketing or not.

It's not like alternatives to Bayer patterns, like RGB stripes, don't have their own issues. Even using three separate sensors for RGB and using a prism block to split the light has its disadvantages.

Red learned a long time ago not to confuse measurable resolution with recorded pixel dimensions, however, the basic problem is language, not marketing -- "4K" is a shortcut term, few people constantly want to be more specific and type "measurable 4K per color", or "4K RAW", or "4K RGB", whatever. It's not generally done to obscure or mislead.

When Dalsa first announced a "4K" camera, they spent a lot of time arguing with people who said that a Bayer pattern meant that they only had 2K for green and 1K each for red and blue. The mistake Dalsa made was insisting that their algorithms were giving them 4K per color rather than being clear that they meant that the debayering algorithm created 4K files per color channel but that measurable resolution would be slightly lower. To some degree, Red just got caught in that ongoing argument. But it's sooo mid-2000's, you know what I mean? It's 2011 and we're still debating whether a 4K sensor is 4K? Why not point out that an 1080P HD recording can't resolve 1920 x 1080 without aliasing?

No one is being dishonest -- John Galt believes his approach is best, Red believes their approach is best, and everyone is free to test the cameras and make up their own minds.

 

[Graeme Nattress]

Old thread here: http://www.reduser.net/forum/showthread.php?t=51550

Graeme

 

[David Rasberry]

What gets lost in this argument is that the human eye sub samples red and blue too. A Bayer pattern sensor is so far the closest mechanical analog to how the human eye quantifies light. The optic nerves and brain are quantization samplers behind a very elegant chemical charge sensor in the form of rods for luminance and cones for chrominance. A nerve cell fires when it reaches a charge threshold.
So the question ultimately is what is visually relevant?

 

[David Mullen ASC]

I'd also point out that each color layer in a film stock doesn't resolve detail to equal degrees.

 

[Vico Martin]

What gets lost in this argument is that the human eye sub samples red and blue too. A Bayer pattern sensor is so far the closest mechanical analog to how the human eye quantifies light. The optic nerves and brain are quantization samplers behind a very elegant chemical charge sensor in the form of rods for luminance and cones for chrominance. A nerve cell fires when it reaches a charge threshold.
So the question ultimately is what is visually relevant?

The retina is bayer?, that´s true?

 

[Harry Clark]

Cones and rods. Cones and rods.
Cheers,
Harry

 

[Graeme Nattress]

It's not cones and rods, but cone the relative distribution of the L, M and S cones. Rods are only used for really dark vision and don't give us colour information.

Graeme

 

[Vico Martin]

Cones and rods. Cones and rods.
Cheers,
Harry

cones and rods in the retina, but with sub-sample?

 

[Tom Lowe]

Lol, are Canon and Nikon tricking people when they say their Bayer-pattern cameras are "21 Megapixels"?

 

[Graeme Nattress]

For cone distribution, here's a good paper I found showing the pattern of the cones: http://www.cns.nyu.edu/~pl/pubs/Roorda_et_al01.pdf it's not quite the same distribution as Bayer pattern, but you get the idea. For normal colour vision, rods play no part.

The answer to the initial question is though: what do the images look like, and how do they measure. RED images appear high resolution with no added sharpening artifacts and very low aliasing / moire. The measured luma resolution from 4k source files is ~3.2k.

Try the same on the F35, the camera that Panavision's Genesis is based on. It measures sub 1920 on the luma resolution, limited by chroma moire. Vertically there's strong luma aliasing.

Here's a close up of the F35 chroma moire:

Now, if the original article had been a balanced discussion on camera resolution we'd have seen some images showing resolution, aliasing and moire for various camera or sensor type systems. When you see the image above, it really does explain why that approach was not taken.


Graeme

 

[Robert Sloan]

We already discussed that article a while back.

Old thread here: http://www.reduser.net/forum/showthread.php?t=51550

Graeme

Ah.. My bad. I tried looking for it, but was obviously unsuccessful. I did have a question on the motion HDR:

The RED Epic, and thusly, eHDR or HDRx, does two exposures. One normal(includes the low light protection) and one underexposed to protect the highlights. The "Dynamax" sensor does six exposures and combines them.

Normal, Highlights, and Lowlights are the exposures I've heard of capturing for most HDR stills. Could you go into the decision, or process of deciding how the eHDR and HDRx was set at two exposures? Is doing 6 exposures excessive?

I've seen the sample HDRx clips that Jim has put up, and they look amazing. I'm just curious since the methodologies seem very different.

 

[Graeme Nattress]

Who really knows what Panavision is up to with their "HDR" sensor. When they have sensors working in cameras that we can get images off, we can talk more on what they're up to. AFAIK The F35 combines two photosites to get one as is.

Graeme

 

[David Rasberry]

For cone distribution, here's a good paper I found showing the pattern of the cones: http://www.cns.nyu.edu/~pl/pubs/Roorda_et_al01.pdf it's not quite the same distribution as Bayer pattern, but you get the idea. For normal colour vision, rods play no part.

The answer to the initial question is though: what do the images look like, and how do they measure. RED images appear high resolution with no added sharpening artifacts and very low aliasing / moire. The measured luma resolution from 4k source files is ~3.2k.

Try the same on the F35, the camera that Panavision's Genesis is based on. It measures sub 1920 on the luma resolution, limited by chroma moire. Vertically there's strong luma aliasing.

Now, if the original article had been a balanced discussion on camera resolution we'd have seen some images showing resolution, aliasing and moire for various camera or sensor type systems. When you see the image above, it really does explain why that approach was not taken.


Graeme

Thanks for the article link. Very interesting.

 

[Robert Sloan]

Who really knows what Panavision is up to with their "HDR" sensor. When they have sensors working in cameras that we can get images off, we can talk more on what they're up to. AFAIK The F35 combines two photosites to get one as is.

Graeme

Fair enough.

 

[Michael Olsen]

I am curious; the megapixel statement above made me do a little bit of thinking, and I realized there are a few questions I have.

Obviously for still photography, a single-chip CMOS sensor is now pretty much the norm. Canon and Nikon use them (exclusively?) in their professional range of cameras. Likewise, unless I'm mistaken, do Pentax, Leica, etc.

Question 1: Do these cameras use bayer mask CMOS sensors? Or do they use some other arrangement?

Now, I'm aware that there are some specialty sensors out there - Foveon, etc. And I am also aware that Medium Format cameras a viewed as a "highest quality" alternative.

Question 2: When shooting MF digitally (or with a digital back), is the array still CMOS? Is it bayer?

There are a great many individuals, professional and otherwise, who seem willing to basically argue the necessity of 4:4:4 YUV color to death. But I've never heard an ounce of anyone discussing color depth in photographic terms, outside of bit depth.

Question 3: Is there a photographic equivalent of "4:4:4" color that provides a supposedly "truer" representation of all of the color available?

My understanding is that 4:4:4 is essentially a container, not a specification (in the same way that 1080p provides a visual field of 1920x1080, but does not mean a camera actually resolves all of that space). That being said, it only makes sense that some cameras which offer 4:4:4 output are really not recording material that would measure out against a prototypically perfect 4:4:4 image.

Question 4: Is there a way to measure YUV...accuracy? or is it better to say resolution? For example: "Camera A uses an oversampling bayer CMOS sensor to record in 10-bit 4:4:4. Due to inherent limitations, however, the peak color data is really closer to 3.8:2.5:0.8."

 

[Matt W.]

Question 1: Do these cameras use bayer mask CMOS sensors? Or do they use some other arrangement?

Question 2: When shooting MF digitally (or with a digital back), is the array still CMOS? Is it bayer?

Question 3: Is there a photographic equivalent of "4:4:4" color that provides a supposedly "truer" representation of all of the color available?[/COLOR][/B]

Question 4: Is there a way to measure YUV...accuracy? or is it better to say resolution? For example: "Camera A uses an oversampling bayer CMOS sensor to record in 10-bit 4:4:4. Due to inherent limitations, however, the peak color data is really closer to 3.8:2.5:0.8."

File size is a much bigger deal with video than it is with film. The reason these "containers" (4:4:4 or 2k or whatever) exist is because there needs to be a standard format that's useable as part of a post workflow and you need to get the most out of it. With stills, you can get a huge file and zoom in and out in Photoshop and that's okay; with video, you want to work at 100% in real time and without HUGE volumes of storage. That's why 3CCD cameras and whatnot mattered and everyone was format crazy with SD. You only had 300,000 pixels to work with--you had to make sure they were good! With digital stills, 20 megapixels for instance isn't that much harder to manage than 10 megapixels; just install more ram. And, to be honest, 20 megapixel photos don't look that much better if you compare prints side by side. It's largely marketing. But just like SD or 4:2:2 1080p or whatever, 2K is a container, and so the source material (film, or Red, or Genesis) may be starting with way more pixels and then for post you're trying to fit them into a small container as best possible. (Significantly, the 2K container holds the vast majority of film's apparent sharpness even if film can resolve nearly 4k with very low contrast at that resolution; also significantly, the Genesis has like 12 megapixels on its sensor for 1080p output.)

Virtually all dSLRs (excepting Foveon and maybe a few others) use Bayer sensors and OLPFs (excepting the Leica M9, which really ought to have one). Virtually none of these cameras are very sharp on a per-pixel basis. Imagine cropping a 1920X1080 chunk from a 7d and using that for film out. Yuck... Medium format digital backs are Bayer CCDs without OLPFs. The theory is your focus is so shallow only a tiny portion of the image that's in focus will fall prey to aliasing or you're diffraction-limited. Never used a MFDB, but uhh...sounds a little questionable. But professionals put up with way more shit than amateurs would ever be willing to, so uhh...who knows. And because the sensors are big and the lenses are awesome these things really can produce amazing results.

Measuring in terms of "containers" (4:4:4; x number of megapixels; x Ks or Ps) is maybe possible but foolish. There is a measure of sharpness, and a good one: mtf curves. They're simple to read and figures are available for most films. On one side of the axis is cycles (line pairs) per millimeter, on the other is contrast (on a scale from 0 to 1) and the graph plots how good the contrast is for different amounts of detail and for different colors. Unfortunately, most digital sensors do not have mtf curves easily available. Reading mtf curves, one conclusion becomes obvious: film can record very high frequency detail, but that detail is mush and matters very little with respect to apparent sharpness compared with coarse detail.

Marketing departments love that 20 megapixels sounds twice as good as 10 megapixels. In practice, the difference is minor. And in practice, a lot of information can fit into a 1080p or 2k container. (Or only a tiny bit, in the case of the 7D, which is barely better than 720p and worse in terms of color resolution, and I say this despite liking that camera a lot.) What matters most is whether the cinematographer and director make the visual information worthwhile in the first place. A 2k container is actually pretty huge in terms of apparent sharpness stored.

 

[Graeme Nattress]

http://www.reduser.net/forum/showthread.php?t=41929&highlight=foveon

Foveon needs NR on the chroma, and suffers from issues to get good colorimetry as it's hard to use silicon as a color filter.

more later...

Graeme

 

[Graeme Nattress]

MF backs don't generally use OLPFs. Making an OLPF that size is hard and expensive, and MF shooters spend time in Photoshop removing moire by hand, or just shoot things where it matters not. Some MF raw converter tools have moire removers, but I've not been too impressed with the ones that I've seen. Foveon omits the OLPF and has significant aliasing too.

If you double the mp, you 141% the linear resolution. That's significant enough, but other factors come into play - not least lenses and aperture diffraction.

Yes, you can fit a lot of info into a 2k container, but only from an oversampled source properly filtered and downsampled or you get ugly horrible aliasing as the tests show.

Graeme

 

[Michael Olsen]

Thanks Matt and Graeme.

I now have one or two more questions...

1) If all of these professional still photography cameras use single sensor CMOS systems, are photographers as outraged about their lack of color response as some cinematographers seem to be about the use of Bayer CMOS in video/digital cinema?

2) Why is high MP Bayer apparently the gold standard for photography, but is something that people...look down on for motion picture work?

3) Have there been attempts at producing something akin to a striped+binning sensor for photography? Or the use of prisms? Though I imagine size would become a limiting factor in the latter case.

--

4) My question about 4:4:4 sampling...

What happens, say, if I am presented with a camera that will output 4:2:0, 4:2:2, and 4:4:4 to some very nice codec/format. What can I do to figure out how much color data is actually being recorded - one would hope there would be a difference in image fidelity between each level, but is there an empirical measure akin to MTF, but for color? Or is using MTF for each channel being recommended as a testing procedure?