Welcome to our community

Be a part of something great, join today!

  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

Camera shootout 2011

Status
Not open for further replies.
The slanted edge approach leads to silly results though. We all know you can't have more resolution than a sensor has pixels, but the numbers calculated by that approach result in such numbers as it doesn't distinguish between real resolution and aliases. Jack posted a similar response to Adam Wilt on the PVC article: "The method allows spatial frequency response above Nyquist to be measured, although such frequencies will alias".

The two issues there are the issue of counting aliasing as resolution, which I think we can all agree is a wrongheaded approach, and the 2nd, which is in some ways more serious, which is that figures are presented to an audience without adequate explanation of what they mean, and it's only later when questioned by a careful observer like Adam that any kind of answer is given at all.

And if it's MTF that's being measured - why is the chart labeled "maximum resolution", and why is not an MTF plot provided instead with the area that represents aliasing shown? Even with with the bar graph used, aliasing could have been noted in another colour on the bar to show this, but it is not.

And remember here - I'm complaining about a test that rates the resolution of a RED One much higher than I've measured and quoted to customers. We neither want nor need to show resolution figures greater than we deem possible with our camera systems as that helps nobody, and gives false expectations. So what happens when someone goes out and shoots something and they can't get the resolution quoted - who takes the flak - is it me, or you? If you can sense my frustration here, you're right - I'm frustrated by this.

"so I can't tell you in simple terms why this was the preferred method. " - but I can tell you simply why it's NOT a preferred method :

it produces overly large numbers not representative of what the camera system can practically achieve,
is prone to produce "silly" numbers where the resolution is greater than the sensor pixel count,
counts aliasing as resolution,
and very hard to understand! It's a computational approach rather than a direct measuring approach. With a direct measure approach we can all "see" the answer quite clearly from a good chart, whereas with a computational approach you rely on a lot of math which is not always easy to follow.

Graeme
 
I agree with Graeme. I don't understand the point of changing to this method of measuring resolution.

However, I have one suggestion: I would love it if someone would make a chroma resolution test chart.

Eg in addition to black-on-white trumpets we have red-on-green trumpets, blue-on-yellow trumpets etc. And some secondary colors. A MacBeth chart trumpet version would be fun :)

That would be an improvement worth the effort. RE diagonal MTF - the paper makes a good theoretical case but until they account for aliasing and false detail... I'd say it's not really fully baked. Did I miss something?

Bruce Allen
www.boacinema.com
 
I'm not keen on trumpets as they're hard to read Bruce - it's too easy to confuse aliasing with resolution. But I do use a multi-colour zone plate, with red, green and blue zones for just the reasons you describe.

Graeme
 
I'm not keen on trumpets as they're hard to read Bruce - it's too easy to confuse aliasing with resolution. But I do use a multi-colour zone plate, with red, green and blue zones for just the reasons you describe.

Graeme

Are the plates red against green - or just red against black, blue against black, etc?

I think it would be a good sensor workout to have colors with the approx same luma level against each other, then colors with medium luma difference against each other, then colors with high luma difference. Primary colors, secondary colors, pale and super saturated, etc.

Touche on trumpets BTW. Zone plates of different colors against each other then.

I'm amazed DSC labs has like 9 million different kinds of chart but nothing that tests resolution of one color against another of similar luma?

Bruce Allen
www.boacinema.com
 
The shootout already failed on DR/Latitude test just because they fooled themselves showing adjusted results that everybody can see it.

I showed that fact here in this tread before.
 
If you test colours of similar luma as resolution next to each other, you're in a situation that's quite un-like anything I can think we'd shoot as it's really rare to have a luma edge transition without a corresponding colour change, or a colour change without a luma change. I have one shot where a small area has some orange and green in such a pattern, and that's the only real-world example I've seen in my years of collecting RED footage.

DSC do do a chart with various colour combos on zones - perhaps not the ones you want though. Alan Roberts has zone plates with pure chroma on them - no luma, and they're quite funky indeed.

Graeme
 
If you test colours of similar luma as resolution next to each other, you're in a situation that's quite un-like anything I can think we'd shoot as it's really rare to have a luma edge transition without a corresponding colour change, or a colour change without a luma change. I have one shot where a small area has some orange and green in such a pattern, and that's the only real-world example I've seen in my years of collecting RED footage.

DSC do do a chart with various colour combos on zones - perhaps not the ones you want though. Alan Roberts has zone plates with pure chroma on them - no luma, and they're quite funky indeed.

Graeme

Thanks, Graeme! I will check it all out.

Bruce Allen
www.boacinema.com
 
Part 2 is up on zacuto.com. I like the storytelling aspect of the documentary as far as making camera tests exciting (difficult task there), but I'm still looking for those white papers. Have they been posted yet?
 
Hi Jared,
I'm still working with people from the SCCE to put together the white papers, they are not available yet.

-Scott
 
Scott, thanks for the quick response and your presence here. It can be tough to face criticism (or take criticism for something you didn't even do...you're only the messenger), so thank you for sticking it out and patiently answering our questions. We're all passionate here about our work and our craft, so emotions can get a little out of hand at times.

Do these white papers exist, or do they have to be written first based off of the data? Do you think we will have them pretty soon after the web series airs, or will it take a bit? Not that I need to have them right now, my curiosity is just getting the better of me. The documentary is more of an advertisement for the raw data to me, as I want to see first hand what others have been seeing. It's the only way to be responsible: report on what I can see first hand.

I appreciate the work that went into this documentary, and I look forward to discussing the data.
 
So, according to the new rez graph in the video, film stock has a physical resolution..... Um.... And still we have measured resolutions being greater than sensor physical resolutions, and even there it's not clear because the DSLR "physical resolution" seems to be the movie container size rather than the sensor resolution. And although we know that any resolution above what a sensor is physically capable of is aliasing, we have cameras where their measured resolution is much less than their physical resolution that also show aliasing (and chroma moire) on the shown test charts. I'd suggest such a graph and measurement of resolution confuses more than it educates, and the associated video commentary does little to help ease my well known concerns.

Graeme
 
The white papers, as of now, don't exist in a coherent form. I have a stack of camera logs on my desk thats about 2 inches thick and 8TB of data on a couple of hard drives. I've been working to make sense of it all to explain it in the show and asking people that were involved for additional information to plug the holes that I can't figure out. AMPAS has a copy of the RAW data that they are working on organizing this summer, and I hope to continue to be able to work with them to help get everything documented.

At the least when when the show is all finished and released, I want to release selects from the RAW data from the tests as torrents for people to play with. I've been trying to compile what selects of the footage would be most useful and then my plan would be to write up the information on the camera logs and put that as a text or spreadsheet along with the raw data.

As far as the methodology of the numbers and the charts used in the SCCE (At least from what I can tell) most of what was done is based on the work of Dietmar Wüller and his company http://www.image-engineering.de/

His website has a lot of technical documents that go into detail about the methodology that you can look up. I'd love to be able to compile all of of this into a digestible white paper and still plan to do so, but as of right now just getting our show out has taken up all of my time.

-Scott
 
After seeing the video, I have to agree with Graeme (always a safe bet anyway). I'm more confused about resolution.

One thing that isn't mentioned about the canon DSLRs, is that as scene detail goes up resolution goes down due to the compression. I saw a video where they took a nice looking scene, walked in a high detail image, and the whole image goes soft. With the test chart and the spice scene, this is exaggerated on the cameras. It's not a good representation of a normal scene. It's good to know the limitations, but in the video, they seem to indicate those cameras are always that soft. It's just not true.
 
So, according to the new rez graph in the video, film stock has a physical resolution.....

Well it was scanned and the scanner has a physical resolution. So in this representation of the film, there is a physical resolution of the system used to present it as a visible image.

-Scott
 
So Scott, the 1150 figure would imply a scanner with 4088 pixels across? The physical number of 1024 on the RED M-X implies a sensor with 3640 pixels across (now that's a strange number...). On Alexa 810 refers to the sensor 2880 across which is correct at least. So, the physical number refers to film scanner, an unknown number that doesn't make sense, sensor pixel count or as in the case of most of the rest of the cameras, the movie container format. In other words, it doesn't refer to the same thing for each camera....

But what are you trying to convey to the viewer by it? What does it mean? And what about measured resolutions greater than that which is possible? It doesn't make sense at all....

Graeme
 
Were these clips shown to the audience on a 1080p projector, 2k projector, or a 4k projector? For sharpness, obviously Red-MX and film are going to benefit. This could have an effect on the audience reaction parts of the documentary.
 
Jared, the question there would also be what downsampling filter was used as I'm sure they didn't show it everywhere at 4k. And given some cameras use in-camera sharpening and others not, it was interesting to hear Primes mention resolution AND contrast - a lower DR image will have higher contrast and perceived resolution, but whereas you can always contrast-up a higher dynamic range image, you cannot do it the other way around....

Graeme
 
Graeme, I also thought that comment was odd. I definitely felt that the comments glossed over having higher resolution as a benefit. Not only for Red-MX, but for film...which is the golden standard we are ultimately comparing to. There was even an audience member that talked about how a "high resolution camera isn't the only thing to look at...(paraphrasing)". Though this comment is true, the only digital camera in the room capable of motion 4k resolution was Red. In a sharpness test, I was a little let down that the largest guns in the room (film and Red-MX 4k) were almost not even mentioned. Makes me wonder if this was viewed on a 4k projector or not.

Not that I think there is some big conspiracy, I just think if I didn't know anything about resolution and sharpness, I don't know if I would have figured out that Red and film shoot significantly higher resolution than 1080p/2k, and if that even mattered or not. I am curious as to how the clips were displayed during these live screenings.
 
Again, I think the Red came out looking very good. The main complaint about Red has been color, but so far I think the Red's colors have looked great.
 
also keep in mind, red color science always improves. who knows if they have a rc3, and rg3 in the works, or a redlogfilm2. compared to what redspace used to look like back a few years, it's like a different camera all together.
 
Status
Not open for further replies.
Back
Top