Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

Camera shootout 2011

Status
Not open for further replies.
Good points Roberto. Let's see what Schwartzman, Scott, Jackson, and Fincher bring to the table. I'm expecting the movies to look fantastic.

Just as an aside, a good DP needs to paint with light, but also work with numbers (a bit less so with the availability of histograms).

Tim
 
I suppose you are right, but I definitely value the result from the juxtaposition of their shots over the maximizing of image quality of the whole.

I'll take both if I can though! :)
 
Red has survived tests before; ultimately there is a lot more real-world footage shot and shown every day with Red cameras that look great and counteract any argument that a test presents.

Besides, wait for all the Epic-Alexa shoot-outs... and you'll see a lot of people who can't wait to shoot on the Epic. Especially if any comparison test is finished and shown in 4K.

I love the picture quality that the Alexa delivers, but honestly, to me it's a TV camera and the Epic is a M...O...V...I..E camera. With the Epic, I can finally get VistaVision resolution and IMAX grain (or lack thereof) in a tiny camera for relatively low costs -- that's an amazing development. For all of that resolution, frame rate choice, low cost and small size, I'm more than willing to compensate for only a one-stop difference in DR with the Alexa (or just turn on HDRx).
 
David Mullen ASC said:
Red has survived tests before; ultimately there is a lot more real-world footage shot and shown every day with Red cameras that look great and counteract any argument that a test presents.

Besides, wait for all the Epic-Alexa shoot-outs... and you'll see a lot of people who can't wait to shoot on the Epic. Especially if any comparison test is finished and shown in 4K.

I love the picture quality that the Alexa delivers, but honestly, to me it's a TV camera and the Epic is a M...O...V...I..E camera. With the Epic, I can finally get VistaVision resolution and IMAX grain (or lack thereof) in a tiny camera for relatively low costs -- that's an amazing development. For all of that resolution, frame rate choice, low cost and small size, I'm more than willing to compensate for only a one-stop difference in DR with the Alexa (or just turn on HDRx).
Click to expand...


Amen.
 
David Mullen ASC said:
Red has survived tests before; ultimately there is a lot more real-world footage shot and shown every day with Red cameras that look great and counteract any argument that a test presents.

Besides, wait for all the Epic-Alexa shoot-outs... and you'll see a lot of people who can't wait to shoot on the Epic. Especially if any comparison test is finished and shown in 4K.

I love the picture quality that the Alexa delivers, but honestly, to me it's a TV camera and the Epic is a M...O...V...I..E camera. With the Epic, I can finally get VistaVision resolution and IMAX grain (or lack thereof) in a tiny camera for relatively low costs -- that's an amazing development. For all of that resolution, frame rate choice, low cost and small size, I'm more than willing to compensate for only a one-stop difference in DR with the Alexa (or just turn on HDRx).
Click to expand...

Agreed all around David. I just hate to see what appears to be a bogus test creating static in the air over Red.

Tim
 
I know a lot of people involved in the test and wouldn't use the word "bogus". They shot the tests and got these results; perhaps some of their measuring techniques are not strictly accurate, but they didn't set out to make anyone look bad, it was done in good faith.

Red has suffered in comparison tests before and while there are many external factors to blame (the way it was posted, the way it was exposed, whatever) the fact that many of the people doing the tests were experienced with shooting and still got less-than-optimal results suggests a problem that needs to be solved, whether through more education on Red's part or simply more experience over time on the shooters' (and post houses') part.

It may be the flipside of the advantages of a RAW workflow; you give people more options, more flexibility, and you also create more opportunities to do something wrong in the process. That's neither a good nor bad thing, I think it's just the nature of the Red technology, it's not "idiot-proof" despite the flexibility of RAW, though Red continues to make the process more foolproof with improvements to their color science, etc. But if five people do shoot-out tests and three or four of the five don't get optimal results from the Red, it suggests a problem that needs to be addressed.

And it will be, Red is always in a state of improvement, which is great, but people have to keep up with the developments otherwise they are making judgements on technology that doesn't exist anymore.

Of course, the other problem is that most of these shoot-outs are finished at 2K/1080P and some are shown on the internet at even lower resolutions, which basically makes Red's biggest asset -- resolution (well, that and value-for-money) -- invisible to the viewer of the test.
 
David, I didn't mean to cast aspersion on Bob Primes or the Zacuto guys, or anyone else involved in the test. I am sure they were very well meaning, and I complemented participants in an earlier post. Agreed on all the values of Red MX and RAW.

Maybe bogus is the wrong word. I didn't mean to say that the participants purposefully skewed the tests. Perhaps inconclusive, confusing, or poorly interpreted test results would be a better way to put it. Over time there has just been an awful lot of carelessness when it comes to throwing DR and resolution numbers around that could have a deleterious effect on both Red and me as an owner / operator. Makes me nervous.

Thanks for all your contributions to this user group, by the way.

Tim Eaton
 
David, do you think their choice of shooting the MX at ISO 320 would account for the decreased stops in dynamic range for this test?
 
I don't know if that's what they did, how they did it, how they measured it, etc. I keep missing the screenings of the test so I can't comment, but that big of a difference with the Alexa seems a bit off just from my rough experience with both cameras, though I've never measured the difference in DR myself so who am I to judge? In theory, by rating the camera at 320 ASA (if that's what happened) the DR should not have changed, just the clip point would have been shifted.

Truth is that my testing methodology is usually even less scientific than most, I just want to see how these cameras perform in typical and then extreme situations -- I figure that most of the pro cameras are going to look good if shot in optimal conditions... we all remember how good the early Sony F900 demo films looked after all.

I've shot maybe half my 35mm movies on Kodak and half on Fuji, but after 20 years or so, I have yet to shoot a comparison test! Shoot-outs can be informative but in reality we rarely are faced with a situation where we can choose between anything ever made and only have to choose the best technically -- instead, there are a lot of factors that come into play, like cost, the size of the camera, the type of post planned, the release format, the aesthetics of the director, the needs of the story, logistics, etc. so you juggle many issues when coming to a decision, and sometimes serious testing only happens after you've eliminated some contenders, just to simplify your choices and reduce the amount of testing.

On the other hand, the more "shoot-outs" the better... one shoot-out can be misleading, but with several, you can start to see patterns that repeat themselves despite what the shooters are doing right or wrong, certain characteristics emerge.

I've always felt that no matter what anyone says or does, over time, the truth comes out, an accurate picture emerges due to viewing the sheer bulk of material coming out of that particular technology -- on TV, in theaters, on the internet, etc. If you only see one thing shot on the Red One and it looks fantastic, you will form one opinion, or if you see something badly-shot, you'll form another opinion... but it's better to wait until you seen dozens, if not hundreds of projects shot on the camera before you solidify your opinion. Of course, I'm speaking as an observer on the outside, obviously a private owner shooting every day with his Red One is going to quickly get to know what his camera can do. And the other twist is that Red keeps improving their conversion process and old footage keeps looking better... in the long run, perhaps the only opinion you can be sure of is that things will only get better over time.

Basically, with technology changing so quickly, one has to keep an open mind and be ready to change it as the technology changes.
 
I'm beginning to see these cameras (or sensors) more like different filmstocks. Epic - filmstock A, Alexa - filmstock B ... etc. Their looks are different. What is great is they put out nice images.

These test are fine (if you view them in HD or on a big screen) but short lived. Now we have EPIC, soon Sony F65, soon Alexa 4:3 (anamorphic 1.3 for 1:78 or 2x for scope, using more sensor height) and hopefully Aaton digital. Pretty cool.


Robert Torrance
Robert Torrance Camera - Burbank
 
The Red was definitely the cleanest in the shadows... Alexa not so much, looks to be more highlight biased. Overall, I'd declare film the loser... even over DSLRs. Film may have had the best highlight latitude here... but man, does the image look lousy in comparison to any of these cams (maybe not the Phantom Flex though). How do people rate film at over 4K!!!? I just don't see it. Maybe the resolution of "film grain" itself is around 4-6k, but the sharpness doesn't look like anything over 720p, it looked very close to the DSLRs. I also don't understand why people are jumping all over the F3, the highlights were worse than the 7D, which I've been saying for a few months now... but people are just living in denial land. It was really clean in the low-light test, but overall for the price? I think I'll pass. Seems like it's just another "elitist" camera along with the Alexa.
 
I hope to see a DR test shot with the "expose for highlights" methodology one of these days. The procedure would be simple(ish):

1. Set up a very high contrast scene.

2. With each camera, set the exposure so that a given reference point in the scene is just at the brink of being clipped by the camera's sensor. Forget the ISO, forget light meters. Use the in-camera tools to decide this exposure. For the cameras like DSLR:s and traditional video cameras, use as low contrast color profile as possible. If film is included, expose it with half-stop increments to find this point in post.

3. Then try to see how much usable detail can be obtained from the deepest shadows in color correction: how noisy is it, is there banding, compression errors etc. Are the colors usable? What happens if you push the exposure by a ridiculous amount in post, say 6 stops or more?


To me personally, this would be the only sensible way to test the real life dynamic range of the cameras.
 
That may work for still photography, but here's the problem with that in moviemaking: if you base the exposure of each shot in a sequence on the brightest element in the frame so that the clip point is always the same, then your base noise level is going to be different in every shot -- it would be like one shot in a scene being shot at 400 ASA and the next one being shot at 1600 ASA just because there is a bright window in that angle. The truth is that most photography of a scene tries to shoot every set-up at a similar noise level for the sake of continuity -- basically you expose for the subject, which is usually in the midtones, and that's why you need a good DR to hold the brightest parts of the frame. Of course, you may slide the exposure slightly to hold more highlight or shadow information when it's important, but generally you don't radically change exposure shot by shot just to place the brightest highlight at the same exposure point.

But as for testing various cameras for DR, I don't see why it matters if you expose more for the brightest or the darkest thing in the frame, in theory, the DR is the same, it's only the clip or crush point that changes. That's why the best real-world test is something like Robert Prime's lightbulb test where the subject walks from being several stops underexposed to several stops overexposed -- if you cheat the exposure for the highlights you pay for it in the shadows.

But it's true that too many people judge a DR test simply by looking at when things clip, which is not correct since simply by exposing the image a little differently, you can change the clip point. On the other hand, the clip point is often easier to see than the crush point in the shadows where how black to make the blacks is open to debate. In this case, the only really accurate test is something like the Stouffer grey scale test where there are more grey steps than the DR range so you can see exactly when detail appears in the low end and when it clips in the high end, and then debate where to set the minimal acceptable noise floor for black.

Not having seen the demo, it sounds like the Red image was biased for better noise, less highlights, and the Alexa image was biased for highlights but more noise, which may account for the wider DR difference than most people find.
 
They mentioned in the test that each company was contacted to send a technician. Did somebody go from RED? If so maybe they should chime in. If not who did they use? One person was assigned to get the best out of each camera. Surely with all the talent they could find someone who has experience operating a RED to its highest potential. And if they couldn't find someone technically up to the task does that not say something about how difficult a RED is to produce footage, because that is one prevailing perception outside of this forum.
 
Hi guys I too have just seen this yesterday and am a tad angry at these findings. 11 stops DR for MX? You've got to be kidding me. How is it 11 stops yet in the high contrast scene that the female ASC DP (forget her name) lit, the MX held the highlights not only as well but in my opinion even better than the Alexa. The key is look at the woman on the right's hair, on the f35 and Alexa the highlight / rimlight on the hair is coming close to blown whereas on the Red it's very controlled and filmic. Secondly look at the fake "sun" created by some sort of HMI lamp. On the Alexa, there is a sharp halation there (perhaps giving credence to someone's statement here that Alexa footage had sharpening added for some reason?) whereas on the MX it is a smooth light.
Lastly, I am miffed about a seeming bias either towards the Alexa or simply away from the MX. Why is it that the MX lookd the best in the high contrast scene (or at the very least equal to the Alexa/F35) yet they said something along the lines of "Oh the F35 and Alexa both looked great" and completely left out the Red despite the fact that Red was the ONLY other digital cam able to hold the +5 stops overexposed highlights out the window. So how is it the MX held those +5 stops over exposed highlights yet it's 3 stops less DR than Alexa?
Something is rotten with that DR test.
Is there anyone that can do another correct MX DR test to settle this once and for all I'm tired of this false information, I had a friend email me today after I sent them the test and asked me why the Red's dynamic range was so low and comparable to that of DSLR's etc and I had no explanation as of yet but I was pretty certain MX had at least 12.5 to 13 stops.
 
David Mullen ASC said:
That may work for still photography, but here's the problem with that in moviemaking: if you base the exposure of each shot in a sequence on the brightest element in the frame so that the clip point is always the same...
Click to expand...

The test would show how much mid tone and shadow underexposure is acceptable to protect the highlights. That could be used as a rule of thumb when exposing in real life: i.e. whether or not Red MX (or some other camera) could be rated at ISO1600 not just for the shots with extreme highlights, but for the whole scene without worrying about noise.

Also, noise can be matched between shots relatively easily in post (either by adding noise or removing it, or both), but nothing can be done to clipped highlights.

But as for testing various cameras for DR, I don't see why it matters if you expose more for the brightest or the darkest thing in the frame, in theory, the DR is the same, it's only the clip or crush point that changes.
Click to expand...

It matters because clipping truncates the dynamic range of the real life scene that is being captured. If the highlights are clipped, having tons of footroom in the shadows doesn't help, some dynamic range that existed in the original scene is lost for good.

But it's true that too many people judge a DR test simply by looking at when things clip, which is not correct since simply by exposing the image a little differently, you can change the clip point.
Click to expand...

Yep. And they create a lot of fuzz on the interwebz based on that false perception. Me not like.

On the other hand, the clip point is often easier to see than the crush point in the shadows where how black to make the blacks is open to debate. In this case, the only really accurate test is something like the Stouffer grey scale test where there are more grey steps than the DR range so you can see exactly when detail appears in the low end and when it clips in the high end, and then debate where to set the minimal acceptable noise floor for black.
Click to expand...

Why not do this test on a real life scene? That's all i'm asking ;-)

Not having seen the demo, it sounds like the Red image was biased for better noise, less highlights, and the Alexa image was biased for highlights but more noise, which may account for the wider DR difference than most people find.
Click to expand...

Exactly. The real test of the camera's capabilities would be to see the low light / noise performance in a case where the highlight clipping at sensor level is at the same point. Based on the MX footage (quite a lot) and Alexa footage (just a few shots) i've worked with, it might change the perception people have quite a lot - Red MX has very, very low noise, Alexa seemed a bit noisier (as said, i only have a worked on a few shots so my material is narrow). I'd say the MX results are very much acceptable (to me) noise wise even at ISO 4000 and up, if the result is full debayer to 2K / HD size.
 
Peter Dmitriyev said:
Hi guys I too have just seen this yesterday and am a tad angry at these findings. 11 stops DR for MX? You've got to be kidding me. How is it 11 stops yet in the high contrast scene that the female ASC DP (forget her name) lit, the MX held the highlights not only as well but in my opinion even better than the Alexa. The key is look at the woman on the right's hair, on the f35 and Alexa the highlight / rimlight on the hair is coming close to blown whereas on the Red it's very controlled and filmic. Secondly look at the fake "sun" created by some sort of HMI lamp. On the Alexa, there is a sharp halation there (perhaps giving credence to someone's statement here that Alexa footage had sharpening added for some reason?) whereas on the MX it is a smooth light.
Lastly, I am miffed about a seeming bias either towards the Alexa or simply away from the MX. Why is it that the MX lookd the best in the high contrast scene (or at the very least equal to the Alexa/F35) yet they said something along the lines of "Oh the F35 and Alexa both looked great" and completely left out the Red despite the fact that Red was the ONLY other digital cam able to hold the +5 stops overexposed highlights out the window. So how is it the MX held those +5 stops over exposed highlights yet it's 3 stops less DR than Alexa?
Something is rotten with that DR test.
Is there anyone that can do another correct MX DR test to settle this once and for all I'm tired of this false information, I had a friend email me today after I sent them the test and asked me why the Red's dynamic range was so low and comparable to that of DSLR's etc and I had no explanation as of yet but I was pretty certain MX had at least 12.5 to 13 stops.
Click to expand...

Valid questions Peter. I've said it before, but it's worth saying again - the only tests with cameras worth doing are the ones you do yourself. I've seen some glorious mistakes from camera tests over the years - although the "resolution beyond the number of pixels on the sensor" takes the biscuit.

Graeme
 
Not having seen the demo, it sounds like the Red image was biased for better noise, less highlights, and the Alexa image was biased for highlights but more noise, which may account for the wider DR difference than most people find.
Click to expand...

I think there are a couple of clues in the shootout video that indicate this may have been the case. On first viewing the shootout video, what surprised me about the MX "underexposed" scene was how blown out the highlights were compared to the other cameras. The vase, screen left, the window, and the phone (although this could be kick angle) were all clipping or on the edge of clip, compared to Alexa and the film cameras. This means that exposure on the MX was likely set to protect shadows in this scene. As we know, the Red engineers achieved higher ISO and increased DR in the MX primarily by lowering noise in the shadows. When I first got my MX upgrade back, it was surprising to me how deep the sensor could dig into the shadows without incurring a noise penalty. This takes a little getting used to. So it may be that whoever was setting exposure on the Red MX in the shootout was too conservative, protecting the shadows, and not taking advantage of the extra DR available on the MX sensor because of its ability to dig deep with little noise.

The second clue is that the video clearly states that camera settings did not change from the "underexposure" scene to the "overexposure" scene. What we see in the "overexposure" scene is that MX does a very good job of holding the highlights. However, if a less conservative stop had been used for the "underexposure" scene, Red could have presumably held even more of the highlights while still retaining shadow detail without noise. Voila - greater dynamic range.

The third clue is that if the cameras were indeed set at 320 ISO, and that setting was used to monitor the histograms and/or in conjunction with a light meter, then the two clues above are exactly what we would expect to see. You would simply miss the sweet spot of the sensor, which according to Red, moved with MX from 320 to 800, you would not be taking advantage of low noise in the shadows, and you would not avail yourself of an extra 1 1/3 to 1 1/2 stops of dynamic range, and possibly more depending on how ballsy you want to be.

Of course this is a bit of conjecture, albeit based on the video Zacuto released, which is why I said in an earlier post that in any test of this magnitude and potential importance for users/owners of Red and the other cameras, a written record of the methodologies including camera settings should be available. Seems only fair, and it would make the video information much, much more useful.

As an aside, I think one reason why some DPs may have a diminished opinion of dynamic range on the MX sensor is that availing yourself of the maximum DR requires you to dig very deep into the shadows, which is pretty counterintuitive for any DP who cut his teeth on film negative.

Tim Eaton
 
Status
Not open for further replies.
Back
Top