RED V-Raptor [X] 8K VV Dynamic Range Analysis Plus

[Phil Holland]

Hello all,

Here's the results of those first precious yet tedious weeks of testing the new camera.

After about 25+ years of professionally testing cameras, I'm going into more depth than I ever have publicly because there's been a lot more interest in all of it. And it's time to expand on a few things I imagine as so many more are dabbling in these dark arts now.

Here's a look at Dynamic Range, Latitude, Image Noise, with some thoughts on best practices, workflow insight, and a few handy charts.

This took some effort to make pretty and not just all spreadsheets. I hope you find it useful.

25 pages in PDF or JPG.

https://www.phfx.com/articles/RED_V-RaptorX_DynamicRangePlusExtras/

 

[Mike P.]

Any word on if they're bringing Highlight Extension processing/combining (in post/SDK/RCXp) to Legacy HDRX footage?

Even if you had to stick with +3, at the very least it would improve IPP2 compatibility of HDRX.

 

[Dustin Schmidt]

Any word on if they're bringing Highlight Extension processing/combining (in post/SDK/RCXp) to Legacy HDRX footage?

Even if you had to stick with +3, at the very least it would improve IPP2 compatibility of HDRX.

I do not believe there is any chance this will happen. The new Extended Highlights mode is a combination of both the new hardware in the V-Raptor X and the modern software.

 

[M Harvey]

Thanks for making this available, Phil! I know that this represents a huge amount of work, I think it's a great service to all of us.

 

[Les Hillis]

Awesome resource.

Really good presentations and explanations of what's going on with the V-Raptor X.

Really good example too of your skills at translating the results of your thorough and painstaking top-quality testing and research into practical and useful information.

Thanks for sharing it with the rest of us.

 

[Karim D. Ghantous]

Thank you very much for taking the trouble to produce and compile all this data. I'm amazed at the results with EH. 20 stops is nuts. Even without EH, we're seeing 17 stops or so, which is not far off from the Alexa 35, IIRC. I very much look forward to reading the full report.

 

[Phil Holland]

Cheers all. Yes, this one took a bit of head scratching to keep it from being a huge pile of spreadsheets. All of my tests turned out a bit over a 125 pages, so this is just the Dynamic Range, Latitude, and Image Noise related things that I do think is the most useful for image makers.

To answer a couple. The Highlight Extension found in IPP2, which is where we get that extra luminance and color information beyond clip I don't think ever comes to Legacy or Original Color Science. No advantage in it either as far I can tell. Everybody should be on IPP2 w/ REDwideGamutRGB and Log3G10 from my opinion and you can go to REDlogFilm if you need your Cineon Log from there.

Extended Highlight Mode is that new mode found in Raptor [X] which is indeed the evolution of HDRx. There's a lot of processing power going on to pull that off in camera, unsure if it can come to the other Raptors. But I will say the global shutter is helping with the "moment" aspect of it currently. It's a hard thing to quantify, but HDRx and Extended Highlights are fairly different things in practice. The temporal stuff going on in camera is also a bit of something I can't fully get to the bottom of, but I still have Monstros to test against. I can only theorize on the full ways this works without knowing the actual code. And I think longer term, additional improvements in EH processing of R3Ds will be pretty powerful. And it's pretty good already.

In term of older cameras and HDRx, I can see some potential improvements, but I don't think you'd get the full capability of what EH is doing in this case. It's fairly hardware dependent when you look at the footage.

 

[Mike P.]

I do not believe there is any chance this will happen. The new Extended Highlights mode is a combination of both the new hardware in the V-Raptor X and the modern software.

No doubt the new global shutter based capture is hardware dependent and done in camera (which likely can't be replicated on older cameras... maybe Raptor/Komodo-X, but certainly not DSMC1/DSMC2). I'm talking specifically about how EH is combining the two captured A and X tracks in post (after capture), resulting in improved IPP2 compatibility and (presumably) a much better DR mapping. With HDRX under IPP2 there is some wonkiness that typically needs to be handled manually/customized in order to make it a more functional mapping (which gets even worse if you try to use any colour managed workflow).

In any case, motion artifacts aside, there doesn't seem to be a reason that it couldn't be done... I also doubt the motion would look *worse* than "Simple"/"Magic Motion" on rolling shutter/legacy HDRX, but that's another story. Currently you typically have more success using RLF than IPP2 with HDRX (unless you're combing manually, which is obviously a lot more work), and it sucks losing the perks of IPP2 (demosaic, out of gamut colour handling, etc etc) merely because HDRX's mapping hasn't been updated since IPP2's introduction.

 

[Karim D. Ghantous]

I haven't read the report yet, but I wonder if you have done, or will do, a comparison between Raptor X and the medium format sensors? I imagine that the Raptor X will have more DR even without EH.

And on a tangent, I have no doubt that black and white film can give 20-25 stops of DR. All you have to do is add layers. Vision3 has like 17 layers, whereas T-Max has two. Just add another 3-4 layers and there is your 25 stops. In theory, I mean.

 

[Phil Holland]

I haven't read the report yet, but I wonder if you have done, or will do, a comparison between Raptor X and the medium format sensors? I imagine that the Raptor X will have more DR even without EH.

And on a tangent, I have no doubt that black and white film can give 20-25 stops of DR. All you have to do is add layers. Vision3 has like 17 layers, whereas T-Max has two. Just add another 3-4 layers and there is your 25 stops. In theory, I mean.

Lots of topics there. For mirrorless bodies you have motion versus still performance, which is also impacted by the codecs and DSP. A broad generalization is most modern mirrorless cameras land slightly below 15 stops TCDR in still mode. Less or equal to that in video modes. I'd say a good range is 12-14+ stops with 14+ being found on the better cameras. This is true for all current digital formats up to digital 645, 44x33, 135, and below. SNR is all over the map based on the DSP and signal processing. Which is easily seen in the waveform when shooting test charts.

The bigger digital cinema cameras have access to more resources in terms of hardware, power, cooling, etc. You take a system like Raptor class bodies with both heating and cooling technology, it's not exactly the same world. There's additional nuance in the image processing across these cameras as well, which I mention in the doc. Lots of other variables like rolling shutter come into play.

Fortunately I indeed tested all 3 Vision stocks literally when they were all hitting the market. Kodak has been fairly dead on with Vision3 with a maximum dynamic range of around 15 stops. However, Vision3 negative film does have "features" as it's a different medium. i.e. It can be overexposed fairly well which impacts how the image reveals itself from the base fog of the film to the highlight roll-off. In practice motion picture film, depending on the stock, is around 13-15 stops DR. The stops below 18% gray is where film suffers the most. Once you are in the fog, there's nothing there.

The thing that has always been on my mind since the advent of quality digital cinema cameras even when digital wasn't up to snuff was the sort of opposite way they work. Film is forgiving in the highlights, absolutely a ruthless monster in the shadows. Digital is less forgiving in the highlights, once clipped, it's pretty much gone, but done right, there is a nice detail descent into the noise floor and you can dig a bit to get more out in extreme situations. There are some who feel they can't overexposure film, but in reality, it's how overexposure occurs in film as it's very literally a different way of seeing light. Pushing and pulling the film is another interesting technique to increase contrast or decrease contrast of the characteristic curve. Lots of minutiae to get into with film.

SNR of digital versus film at an equivalent sensitivity, heck even 1-4 stops higher sensitivity with digital, reveals digital is much cleaner overall. And SNR with film is depending on the film stock, the negative size, and development/processing. Good news is no matter the format size, dynamic range is the same. 50D is the cleanest current motion picture film, 250D is more or less the best reference stock, 500T is the noisiest. I see it pushed a bunch, up to 3 stops. Generally you don't pull film below 1-2 stops or you get a real flat image.

To make note to your concept with more layers, you become less light sensitive. You can find very, very slow/low speed stocks that are very clean, perhaps more DR, but it's all about exposing light/burning the neg to get out of that base density. Most aren't looking to film anything in motion with those. Most DPs struggle below ASA 50 when lighting interiors.

I guess since we are talking film, Vision3 versus say Portra are different things and different worlds. Reversal film is an even more different world, and far less dynamic range. But film stocks present different aesthetics, color response, and just different character in both color and black and while. That's before we get into the printing process for motion picture film as well.

On thing that is notable and similar to know about is the impact of exposure and flare on the image as well. Underexposed film = more noise, same for digital. Over exposed can left the shadows, actually something people really used to work hard to avoid. The term milky shadows was born out of that. With post color grading technique, less of a concern, but when processing films through mainly the lab, a lot of things you needed to be much, much more careful about.

 

[Phil Holland]

Ah. I guess I should make note if that wasn't clear above. Generally the ideal minimum dynamic range for filmmakers is 13 stops, 15 stops I consider good, anything beyond that could be considered excellent.

I'm certain I've posted some thoughts about digital versus motion picture film with the 7 up/7 down metering mentality. That's how I like to work with film a lot of the time. But for film I am very, very careful when metering under 18% gray 3-4 stops. The knee to toe transition and characteristic curve is steeper, more accurately, film's response to light, is different than digital.

 

[Blair S. Paulsen]

Props to Phil for the detailed DR analysis charts, tables and notes. Worth the read. Huge thanks for the effort.

Personally, I learned a lot. In particular, about the SNR characteristic at various "push" levels. The first few RED models/sensors were much more sensitive to underexposure in the shadows, particularly under tungsten light. Props to Graeme, the sensor designers and the whole RedTeam for getting tungsten performance within a whisker of daylight response. Impressive AF. I was also amazed at how little the noise floor rose even at 3,200/4,000 ISO.

As I suspected from the early frames utilizing Extended Highlights, the chroma tracking into overexposure is (hate this term) a game changer. The saturation may fall off just a bit, but the hue shift is almost eliminated. Crazy town.

Note: because the RED Raptor VV (X) in Extended Highlight mode offers so much luminance and chrominance information in the RAW data file, it should provide a terrific starting point for LUTs and other treatments like in Phil's PHFX toolset. I'd much rather take an image that's a bit too rich and pare it down to fit creative intent vs juicing up a "thin negative". Looking forward to having play.

Cheers - #19

 

[Phil Holland]

CineD has published their Dynamic Range and Latitude test, and it's sparking some conversation.

RED V-RAPTOR [X] 8K VV Lab Test - Dynamic Range and Exposure Latitude | CineD

CineD Lab Test of the new global shutter sensor based RED V-RAPTOR [X] 8K VV camera. We test dynamic range and exposure latitude.

www.cined.com

So, to emphasize what I have written in my rather thorough test, they are showing the result of clipped data from the Xyla. Their reading for Slope-based Dynamic Range is 15.1 stops SNR1 = 14.5 and SNR2 = 12.8.

My results for full RGB exposed, no clip @ ISO 800 = 15.3 stops, 14.3 SNR1, 12.9 SNR2. The 0.2 stop variance is more or less what I called out in my writeup as the lens being a possible variable.

Using that same Xyla if measuring the dynamic range with the chart at ISO 3200 you'll see more of where that Total Captured Dynamic Range is lurking. Which is again, something I show in my results. It's EXTREMELY important to test the full ISO Range for exactly this reason.

I don't know how I can more clearly state that Dynamic Range is not the same as Signal-to-Noise Ratio. SNR describes the "quality" of those stops. Dynamic Range describes how many stops the imaging system is capturing.

SNR can be impacted greatly by in camera processing, downscaling, etc. Dynamic Range doesn't change.

 

[andrewhake]

18.5 stops is pretty nice I guess but you know what would be really nice? 20 stops

 

[Phil Holland]

18.5 stops is pretty nice I guess but you know what would be really nice? 20 stops

My take, which I've been touching on during live presentation for a few years, is between 21-26 stops puts into the goldilocks zone. And obviously how clean and where all that exists is a factor.

And it's not just about having that dynamic range. If done right, you won't need ND unless you REALLY need ND for many situations.

We'll get there eventually, like maybe end of this decade or in the next one. A few major leaps need to occur first. There's a few other tech improvements I want to see as well.

 

[Karim D. Ghantous]

Not news for some, but DxO tested two Red sensors, the Dragon and the Helium. The Dragon matched the 50Mpx Sony medium format sensor in DR. This sensor is used in several MFD cameras. The Helium actually beat it, and still holds the title for best DR tested by DxO. I wonder what the V-Raptor would score if it were tested by DxO?

RED Helium 8K DxOMark Sensor Score: 108 — A new all-time-high score!

RED’s prototype Helium 8K sensor has shattered our previous top DxOMark sensor score. With a dramatic jump to 108, it leaps past the only other sensor to have broken the 100 barrier – RED’s own Epic Dragon prototype (101 points). RED has made a name for itself by producing high-performance...

www.dxomark.com

 

[Shane Williams]

Ok, so problems with the CineD test:

1) Looking at the stair steps, the closest exposure Phil did to theirs was "Test 3 - Plus 1 Stop from RGB Exposed." CineD reports 15.1 stops, 14.5 SNR1, 12.8 SNR2 at ISO 800, and Phil has 16.3 stops, 15.2 SNR1, 13.8 SNR2: a big difference. And this is without either of them subtracting recovered stops.

2) They got +2 max in the latitude test, and Phil and CVP got +3, just like the rolling shutter V-RAPTOR. Another big difference. Also, they have both the V-RAPTOR [X] and ALEXA 35 at -7, while CVP shows both V-Raptors doing noticeably better than the ALEXA 35. The previous ALEXA was about 2 stops noisier than V-RAPTOR, and according to ARRI's website, the ALEXA 35 gains "a stop in the shadows."

3) They don't test the whole ISO range. However, they did test the ALEXA 35 at EI 3200 and got 16.7 stops, 16 SNR1, 14.9 SNR2. They say in their article about V-RAPTOR [X] that, compared to the ALEXA 35, it's "not as consistent across the range of ISO values, as a quick check of ISO6400 revealed: dynamic range drops to 12.2 and 13.7 stops at (SNR = 2 / 1)."

I find it odd that they didn't mention the DR of V-RAPTOR [X] at ISO 6400, instead referring to the SNR, and that they compared SNR from one camera at 3200 to SNR from another at 6400. Looking at Phil's data, it looks like SNR goes down as ISO goes up, understandably as the noise would be more visible.

Using Test 3 again, Phil got 16.9 stops, 13.9 SNR1, 12.3 SNR2 at ISO 6400. So, pretty close SNR numbers, but again, odd that they didn't mention the DR.

For an apples to apples comparison, Phil's numbers from Test 3 for ISO 3200: 17.5 stops, 14.9 SNR1, 13.3 SNR2. There's the consistency they were looking for with the SNR, and even if you subtract the recovered stop you have the same DR as the ALEXA 35 at EI 800. And downsampled to 4K, you would get a 0.5 SNR improvement, according to Phil's slides.

CineD's numbers for the ALEXA 35 at EI 800: 16.5 stops, 16.3 SNR1, 15.1 SNR2. So, at EI 3200 it gained 0.2 stops, while the V-RAPTOR [X] gained 1.2. The maximum Phil got for Test 3, Tungsten, single exposure, was 17.7 stops at ISO 4000. Subtracting the recovered stop, it would match the ALEXA 35 at EI 3200.

If you only looked at the DR at ISO 800, you would be missing 1.4 stops, which more than makes up for the recovered stop. So, yeah, you have to test the whole ISO range.

---

Phil's tests were much more thorough, and I would like to see him test the ALEXA 35.

Also, in case anybody is confused like I was: I wasn't sure if the different ISOs were different exposures or not, but if you look closely at the squares on slide 15, they're the same photo at different ISOs.

Edit: I just realized that I used Phil's Test 3 because it was the closest match to CineD's stair step waveform, but when I compared to CineD's ALEXA 35 test, I didn't use Phil's closest matching test, which would be "Test 1 - RGB Exposed." So, the numbers for Test 1, ISO 3200 are 17.3 stops, 14 SNR1, 12.5 SNR2. And the max was 17.7 stops at ISO 5000, which is 2.4 stops more than ISO 800.

This demonstrates another major problem with CineD's test, which is that the exposure, the degree of clipping, is inconsistent with their ALEXA 35 test.

 

[Phil Holland]

Thank you Shane. Moderately, some of the reasons I released my more expanded test was to illustrate the importance of testing the Full ISO Range. DXO for instance, an actual lab, clearly knows this, as do most who do this professionally. I've been testing the whole range since digital began for the film industry. We even tested film with push and pull processing. Standard stuff for the better part of 25 years. Members of this forum have seen these tests as well. So nothing new there.

I'm unsure why CineD is doing what they are doing and it's a bit more glaring when you see other cameras tested. It's also clear they aren't cross referencing the exposure information against real world images as that alludes to what I showed in my "clipping behavior slide". What the think is a recovered stop is a not what's going on either, but that's a conversation for another day. But also why I showed exposed and underexposed material for that matter.

Also strange that their latitude test starts overexposed by 1.25-1.33 stops, but that allowed them to gain more in the shadows in that specific test, but sacrificed highlights. You won't see me recommending extreme underexposure or overexposure, hence my 3-4 stops recommendation from the Base ISO of 800 as a rule of thumb. There's more to be dug out of the shadows if needed. RED's IPP2 highlight overexposure helps on the other end of the spectrum if needed as well.

My production year is "live now", but I do plan on doing one more camera test later this year as time permits. But I need to wait for a couple cameras to be released.

I'm hoping I illustrated the variance of SNR across systems as well as it is the most effected by image processing, both in camera and in post. Dynamic Range is the only number that shouldn't change in that respect. I find it extremely shocking that cineD puts SNR readings on a Waveform with clearly visible stops unnumbered.

I'm working hard on a few things behind the scenes to make all of this clearer for the global motion picture industry. But it's going to take a hot minute.

 

[Jon Pais]

Alfonso Parra found that at ISO 800, the Alexa 35 has between 14-15 stops of dynamic range, with 9 stops above 18% gray and just over 5 stops below 18%. This is not the total range the sensor can ‘see’, but the usable dynamic range (pretty certain it’s SNR=2). At ISO 800, how many stops above middle gray does the V-Raptor [X] have at SNR=2?

 

[Phil Holland]

Alfonso Parra found that at ISO 800, the Alexa 35 has between 14-15 stops of dynamic range, with 9 stops above 18% gray and just over 5 stops below 18%. This is not the total range the sensor can ‘see’, but the usable dynamic range (pretty certain it’s SNR=2). At ISO 800, how many stops above middle gray does the V-Raptor [X] have at SNR=2?

SNR2 is not Dynamic Range. I can't stress that enough. It's a measurement of the quality and quantity of noise. Even SNR1 doesn't tell the whole story considering the ISO Weighting across the ISO Range.

At the floor of 18% Gray, you get about 8 stops above 18% Gray on V-Raptor [X]. In my document I outline this as well as other exposure situations.

There are several notable differences between the Alexa 35. At ISO 800 they do have more allocated stops above 18% Gray as they are using a Dual Gain/blend/temporal noise reduction method, but shadow detail and color stability below 18% are a different world compared to RED's methodology. Additionally, if you consider downscaling an 8K image to a 4K image, that's worth investigating as well as a 1:1 format size equivalence if working with both systems. If working with both systems, give ISO 1600 a try on V-Raptor and side by side that. Heck, ISO 2500 as well.

Moderately why you need to test the whole ISO Range. When we do multi-system tests, SNR is indeed useful to understand how to match image texture at various ISO Ratings between cameras as well as examining finishing for 8K, 4K, 2K.