Weapon R3D VS. ProRes (Noise Test with all files for download) - Bonus 4K 60 & 120FPS

[Matt Ryan]

Weapon R3D VS. ProRes (Noise Test with all files for download) - Bonus 4K 60 & 120FPS

Intro

Nick Fury just received his stealth Weapon CF #87, so we decided to run some tests for our own use. We figured it may be beneficial to others in the community so we have made the decision to share the results + all the untouched files straight from camera (both R3D's and ProRes files). What we set out to do was see the noise performance between the R3D and the ProRes file. So what we did is shoot from 250ISO through 12,800ISO in 6KFF in the R3D + ProRes mode to capture simultaneous 4K 422HQ files. This is not a "scientific test" and we did our best to match exposure by adjusting the T stop on the lens with each ISO change. It's by no means perfect but it gave us the info we needed. We decided to perform the test using the STANDARD olpf due to the fact that all new Weapons will be shipped with it and the Raven will have the fixed version. So this would be the most useful example for the widest range of users. This was literally the first time we used the camera so our methods of testing may change down the road. This was literally the first time we booted up the camera.

TEST

Our setup was simple. We used a single Area 48 light blasted straight into the grey chart and color chart sitting atop a pelican case. This gave us an idea of not only the noise profile based on the ISO level, but also how colors react. I personally used this test to make Neat Video noise profiles for each ISO level to use in the future. We used a Mamiya 45mm medium format lens to avoid any lens vignetting or falloff. At 250ISO we began wide open at F2.8 all the way down to F22 at 12,800ISO to keep the exposure consistent. We shot 6KFF with Dragoncolor 2 and Redlogfilm. We used the lowest compression ratio possible while simultaneously recording ProRes.

4K 120fps + ProRes 2K 4444 of my face and chart (2000ISO & 4000ISO)

Just decided to jump in front of the camera and see how 4K 120fps, 20:1 compression, and 2,000ISO and 4,000ISO looked. I was more curious about the detail and sharpness so check out that sexy facial hair :ihih:

Also shot the charts in this setup as well.

4K 60FPS & 120FPS + 2K 4444 ProRes (2,000ISO & 4,0000ISO; 120fps at 1,280ISO)

For the heck of it we decided to test out 4K 60fps and 120fps + simultaneous 2K ProRes 4444. Now, if you're shooting simultaneous, the compression ratio gets pretty high. Like 20:1 high. So if you need high speed, either shoot R3D ONLY or Prores ONLY to avoid such high compression. This was a spur of the moment test, shot at 2000ISO and 4000ISO as the sun was disappearing. This also helped me get a noise profile for Neat Video that I can use on future footage. So this may or may not be beneficial to you, but this is worst case scenario. Again, WORST CASE SCENARIO. You should never shoot 4K at 20:1 compression, ever! Please keep that in mind. We debated on whether to post this or not because there will be those who choose to ignore these notes and discount the camera. But we decided to leave it in. If you drop the ISO to 640, that's about what we saw with our own eyes. So keep that in mind! Go Raiders!

Conclusion

Weapon is a beast. I don't know what they tweaked but in our test, 4000ISO is more than usable and if you feed it with proper light levels, the noise is very minimal. Of course if you go outside at night with no light you will have lots of noise because you're trying to amplify what light is available and thus creating noise. But this should be a given. The 1/2 stop improvement RED mentions may be a conservative estimate. If I'm doing R3D + ProRes, I know my 4K ProRes file will look identical to the R3D in terms of detail and noise. So that is good news. It should save some of us time in making dailies via transcodes after a shoot. Imagine just handing over the files and walking away and your editor gets the ProRes files immediately. Doing simultaneous ProRes does take a slight compression hit, but that is expected. The camera itself not only looks better, it finally has everything located in the right places. I can't say enough good things after our initial first look. Literally, this test was the first time we ever booted up the cam and the first time shooting with it. Looking forward to many more shoots with it! If you are looking to rent a Weapon, contact Nick Fury. He was generous with his time and equipment to do this test not to mention he has the best set of Kowa Anamorphics in the world. No joke. If you have a deposit on Weapon, you should be happy. If you don't, you might want to. This uses the same sensor but it seems like a step up. The new body is a thing of beauty and worth the upgrade, especially with the added features. We controlled the camera via FoolControl wirelessly. Why? Because we can without any additional accessories :yesnod:

From looking at the files it appears the ProRes is not influenced by your compression ratio meaning it has higher detail when shooting at higher compression ratios. This is great news!

Files

I'm warning you now, these files are huge. It took me just over 3 days to upload them. Most are 10 seconds in length so you can profile the noise and get a "feel" for it in motion. Take all the files or only what you want. Each folder is titled for convenience. Every shot includes the R3D and the ProRes file. Metadata can be found in the R3D file but these are all untouched straight from camera and in Redlogfilm so the ProRes file has the most range to play with. This is not a "scientific" or perfect test. It is what it is and we were not attempting perfection. Hope these will be of use to some of you. As a long time lurker, I've been anxious to get a look of any footage I could find so hopefully for those of you like me, this will give you something to chew on.

DOWNLOAD

1. 250ISO-12,800ISO R3D Vs. ProRes files:
matthayslett.com/wp-content/uploads/2015/11/Standard%20OLPF%20(250ISO-12,800ISO)%20*Redlogfilm.zip

2. 4K High Speed Files Vs. ProRes: (Drop the walking shots to 640ISO to see exactly what my eye saw. The camera was better at seeing than my own eyes)
https://www.dropbox.com/sh/v3g6fnb1ds7ok62/AAARQsb7QC0ta4HtCZJ4ow6ga?dl=0


If you're interested in a rental of a Weapon, lenses, or support gear, give Nick Fury a shout.
Nick Fury - Fury Rentals
srgtfury@gmail.com
949-689-5614
http://www.reduser.net/forum/showthread.php?111020-FOR-RENT-CARBON-FIBER-DRAGON-SO-CAL-AREA-CON-U-S&highlight=Rent+Fury

 

[Jake Wilganowski]

Cool

 

[Nick Morrison]

Matt and Nick - you guys rock! Thanks for doing this!!

 

[Raphael Varandas]

O man... thanks!!

 

[Nook Kim]

You're too kind Matt!!

 

[Antony Newman]

Thankyou Matt & Nick, I am sure this will be Very much appreciated by the masses.

AJ

 

[Brett Harrison]

Thanks Matt and Nick!

 

[Martin Stevens]

Awesome post Matt and Nick.

 

[David Battistella]

Looking forward to taking a look at these. Thanks Matt and Nick.

David

 

[Ryan Hamblin]

I have not had a chance to look at these tests specifically, but look forward to doing so. The test Phil did with r3d and prores 4K 422. I felt that the prores was just ever so slightly sharper ( when viewed at 150% ). So really not much difference at all. Color grading the 422 was in my opinion only ever so slightly less gradeable than the r3d. Basically you would really have to eff up your shot to not be able to do a nice grade on the prores. What are you all finding?

 

[Arthur Woo]

First off, to dispel any allegations of being a troll, this post is meant to feed my own knowledge and understanding and does not have any ill intent towards this test, towards RED or its cameras (I love my Epic Dragon).

Matt, or anyone else, would you do me a favor and explain to me usable ISO 2000/4000/etc when fed with proper light levels? Why shoot that high of an ISO if you have proper light levels? I was under the impression the school of thought is to shoot at the native ISO whenever possible since everything is metadata, even though we can change it in post because it's RAW. What practical purpose would shooting at a higher ISO give you if you already have ample light?

This is one area that I haven't wrapped my head around - perhaps because I never shot on film (except in one class), and I come from the DSLR generation and started shooting on Canons and eventually upgraded to Epic, so perhaps my understanding of ISO is a bit lacking.

The way I come to understand judging digital cameras in regards to ISO and 'usable' footage, is by how much noise there is, regardless of the light levels. For example, we see amazing low light cameras such as the a7s ii, the C300, and so on - where you can easily boost the ISOs in camera when ambient light is fading, and get a usable image. However, with other cameras, if you shoot 1600 ISO and beyond with low ambient light, more noise is present in the image.

Now that you know the perspective I am coming from, I'm having a hard time understanding the point of testing 2000/4000 ISO (and what's the technical 'how-to' behind it? Closing the aperture and using ND filters) - just to say that it can? Is it to introduce a certain noise pattern as you would with film? I guess that wouldn't make sense if the idea is to get usable footage. As you can see, I'm confused. Please enlighten me! Thanks!!

 

[AndreeMarkefors]

Arthur,

most cameras out there, when setting a higher ISO on the camera, will apply gain to the signal to amplify it. The problem is that in low light scenarios, the 'light part' of the signal is already low and close to the noise floor, so when you amplify it, you also amplify the noise to a large extent. The "amazing low light cameras" you refer to can use higher ISOs than expected without producing too much noise—but this you already know.

RED cameras do not have the ISO function (applying gain) you've used in DSLRs. They don't really have ISO (in this sense) at all.

Setting your RED camera to a certain ISO is more like a declaration of intent—"when I get my files in RedCineX I will set the ISO slider to THIS". The reason you need to set it in the camera is of course so that you get a visual representation of what that setting looks like when pushed in post.

Changing the ISO on your camera will not in any way determine how much light your image gets. You control that with aperture and shutter (and filters). This very important fact is the reason you want to be very aware of the "goalposts". BUT—setting a certain ISO in camera (rating your camera to that ISO) will affect how you set your aperture and shutter for a certain light level.

PUSHING or PULLING the native file in post (to your rated ISO) will affect the texture/grain of the result, but perhaps more importantly the distribution of STOPS in relation to middle gray.

Forgive this perhaps too elementary breakdown, but I figure this way more people might pick something up.

 

[Les Hillis]

...4000ISO is more than usable and if you feed it with proper light levels, the noise is very minimal. Of course if you go outside at night with no light you will have lots of noise because you're trying to amplify what light is available and thus creating noise. But this should be a given...

Arthur, I don't think Matt was suggesting using 4000ISO when you have ample light, I think he just meant when you have 'proper light levels' for 4000ISO (ie. when using less light than 'proper light levels' for 250ISO, but more light than 'proper light levels' for 5000ISO for example). The 'proper light levels' can be different, but can have (practically) the same result (that is, being effectively equal to having an ample amount of light) when matched to their 'proper' ISO/metadata adjustment.

So, your understanding of how to expose is correct. Meaning yes, people generally try to expose the sensor within it's native/ideal range (and Matt is saying its range extends to 4000ISO, with minimal noise), while the ISO settings can also be used as a visual aid in-camera to help you over and under expose for creative purposes (for adding noise/grain and for getting more or less shadow/highlight detail).

I could be misinterpreting things myself, but I think your confusion is simply the result of equating 'proper light levels' to 'an ample amount of light', when they aren't always the same thing.

 

[Brett Harrison]

Arthur I think the idea is proportional light levels to the ISO, not light levels suitable for higher ISO - which he never said.

IOW, even if you have an A7S, if there's no street lights or moon, you're still going to get massive noise because you can't perform magic with it.

First off, to dispel any allegations of being a troll, this post is meant to feed my own knowledge and understanding and does not have any ill intent towards this test, towards RED or its cameras (I love my Epic Dragon).

Matt, or anyone else, would you do me a favor and explain to me usable ISO 2000/4000/etc when fed with proper light levels? Why shoot that high of an ISO if you have proper light levels? I was under the impression the school of thought is to shoot at the native ISO whenever possible since everything is metadata, even though we can change it in post because it's RAW. What practical purpose would shooting at a higher ISO give you if you already have ample light?

This is one area that I haven't wrapped my head around - perhaps because I never shot on film (except in one class), and I come from the DSLR generation and started shooting on Canons and eventually upgraded to Epic, so perhaps my understanding of ISO is a bit lacking.

The way I come to understand judging digital cameras in regards to ISO and 'usable' footage, is by how much noise there is, regardless of the light levels. For example, we see amazing low light cameras such as the a7s ii, the C300, and so on - where you can easily boost the ISOs in camera when ambient light is fading, and get a usable image. However, with other cameras, if you shoot 1600 ISO and beyond with low ambient light, more noise is present in the image.

Now that you know the perspective I am coming from, I'm having a hard time understanding the point of testing 2000/4000 ISO (and what's the technical 'how-to' behind it? Closing the aperture and using ND filters) - just to say that it can? Is it to introduce a certain noise pattern as you would with film? I guess that wouldn't make sense if the idea is to get usable footage. As you can see, I'm confused. Please enlighten me! Thanks!!

 

[Jake Wilganowski]

Arthur, another thing to know, is RED does NO noise reduction in camera. Virtually every other camera does, ESPECIALLY the ones you referenced. When you use noise reduction, you are doing something destructive to the detail and color channels that can't be reversed. That is why red leaves it up to you to do in post. Play with neat video and you will see how clean red can become at low light levels when processed correctly.

It all stems from red raw being a digital negative with the least possible in-camera processing to have the most flexibility in post.

Really study up on how red works, because it indeed works very differently than any other camera, except film.

 

[Steven Decker]

Dropbox has killed the link already.

Bummer

 

[Marc Wielage]

Good work, Matt. I've said for years that if the R3D's are correctly debayered and adjusted, a ProRes 444 transcode looks just fine. The key is getting the color temperature and overall balance right in the beginning.

 

[Matt Ryan]

First off, to dispel any allegations of being a troll, this post is meant to feed my own knowledge and understanding and does not have any ill intent towards this test, towards RED or its cameras (I love my Epic Dragon).

Matt, or anyone else, would you do me a favor and explain to me usable ISO 2000/4000/etc when fed with proper light levels? Why shoot that high of an ISO if you have proper light levels? I was under the impression the school of thought is to shoot at the native ISO whenever possible since everything is metadata, even though we can change it in post because it's RAW. What practical purpose would shooting at a higher ISO give you if you already have ample light?

This is one area that I haven't wrapped my head around - perhaps because I never shot on film (except in one class), and I come from the DSLR generation and started shooting on Canons and eventually upgraded to Epic, so perhaps my understanding of ISO is a bit lacking.

The way I come to understand judging digital cameras in regards to ISO and 'usable' footage, is by how much noise there is, regardless of the light levels. For example, we see amazing low light cameras such as the a7s ii, the C300, and so on - where you can easily boost the ISOs in camera when ambient light is fading, and get a usable image. However, with other cameras, if you shoot 1600 ISO and beyond with low ambient light, more noise is present in the image.

Now that you know the perspective I am coming from, I'm having a hard time understanding the point of testing 2000/4000 ISO (and what's the technical 'how-to' behind it? Closing the aperture and using ND filters) - just to say that it can? Is it to introduce a certain noise pattern as you would with film? I guess that wouldn't make sense if the idea is to get usable footage. As you can see, I'm confused. Please enlighten me! Thanks!!

Arthur, I don't think Matt was suggesting using 4000ISO when you have ample light, I think he just meant when you have 'proper light levels' for 4000ISO (ie. when using less light than 'proper light levels' for 250ISO, but more light than 'proper light levels' for 5000ISO for example). The 'proper light levels' can be different, but can have (practically) the same result (that is, being effectively equal to having an ample amount of light) when matched to their 'proper' ISO/metadata adjustment.

So, your understanding of how to expose is correct. Meaning yes, people generally try to expose the sensor within it's native/ideal range (and Matt is saying its range extends to 4000ISO, with minimal noise), while the ISO settings can also be used as a visual aid in-camera to help you over and under expose for creative purposes (for adding noise/grain and for getting more or less shadow/highlight detail).

I could be misinterpreting things myself, but I think your confusion is simply the result of equating 'proper light levels' to 'an ample amount of light', when they aren't always the same thing.

Hey Arthur, Les pretty much nailed it. When I said appropriate levels, check out the files to get an idea of what I mean. You will notice at each ISO level the Histogram is exposed almost identically. That would be proper light levels for the given ISO. Proper light levels at 4000ISO is still less light than at 2,000ISO for example. The a7s is designed specifically for clean low light images with its large megapixels and built in noise reduction. Those cams have a certain appeal because they apply gain to the image in camera to "boost" the amount of light gathered, and then apply aggressive noise reduction to counter the noise. A RAW cinema camera like a RED does not apply any in camera noise reduction and carries many more mega pixels which is not the greatest for high ISO. That is why the a7s and c300/500 are in the 12mp range, canon being 8mp I believe. Larger pixels = more sensitivity and less resolution to handle. This makes it not only cleaner (because more light is gathered per pixel) but the added in camera noise reduction further eliminates any noise there would have been there had the signal been captured RAW. If you want to try a test, shoot with the a7s in RAW stills mode and then do video at the same ISO levels right after. Is the RAW still just as clean as the video? No, there's plenty of samples online. A true RAW camera should not apply any noise reduction or sharpening and that is why the video from the a7s is cleaner than its RAW stills.

So again, it doesn't really matter whether you shoot at 2,000ISO or 4,000ISO on Weapon if you expose it properly for that ISO level. Like I said above, it's about getting a full histogram for the given ISO. That is the key to shooting RED. If you were to expose to the right without clipping, you will end up with an even cleaner image. Try it out and see for yourself, because when you do that, you are basically exposing for a lower ISO than you are actually shooting. Give it a try.

 

[Matt Ryan]

Good work, Matt. I've said for years that if the R3D's are correctly debayered and adjusted, a ProRes 444 transcode looks just fine. The key is getting the color temperature and overall balance right in the beginning.

Absolutely. If color temp is nailed, the files are more than good enough to take through post.

 

[Matt Ryan]

Dropbox has killed the link already.

Bummer

They should be back up soon enough. There was nearly 1,000 downloads today so far... Overloaded it a bit