Disappointed at CineD's recent lab test and review of V-Raptor

[Neil Abeynayake]

I'm kinda disappointed at CineD's recent lab test and review of V-Raptor. Here is the url to their report.


https://www.cined.com/red-v-raptor-...hutter-dynamic-range-and-latitude-plus-video/

What do you folks think about this? Are they being biased towards ARRI nad SONY or is RED falling behind a bit which, I hate to believe.

 

[Nick Morrison]

I'm confused. The review they gave is STUNNING. They basically say that the Raptor is incredibly close in performance to an Alexa Mini LF (for exposure), even though it costs what 3-4x less? And has way more FPS?

To even get close to an Alexa Mini LF is a staggering accomplishment for the price:

Our current full frame benchmark for dynamic range is the ARRI ALEXA Mini LF with 13.5 stops at SNR = 2 and 14.7 stops for SNR = 1, without highlight recovery. The ARRI ALEXA 35 (Super 35 sensor) exhibits 15.1 and 16.3 stops at SNR = 2 and 1 respectively (also without highlight recovery).

Judging from the waveform and IMATEST results, I would say the RED V-RAPTOR has about 1 stop more dynamic range than the best consumer full frame cameras. Whereas the ALEXA Mini LF has about 1 stop more and the ALEXA 35 three stops more dynamic range than the RED V-RAPTOR.

And you can't compare the Alexa 35 to ANY CAMERA right now, it's in a league of it's own - and costs as much.

RED is going back to what they've always done - for the $$$, outperform everyone and give indie producers a real advantage. But an Alexa 35 is still in a class of its own and that's OK.

 

[Phil Holland]

With Nick here really.

The shortest statement I can make is the Alexa 35 features a Dual Gain Architecture w/ Temporal Noise Reduction within the imaging path. Essentially an HDR blend of two image signals with image noise removed then re-grained via "Textures" to match both blended signals. This produces several things in terms of ISO Weighting, Calibration, and where stops land at a given ISO.

V-Raptor doesn't do any of that and currently is in "2nd place" in DR based on my tests. Followed only by RED Monstro, Sony Venice 2, then Arri LF in that order.

Considering camera price, form factor, specifications/features, framerates, and format size there's not much to be too grumpy about as it meets or exceeds in those categories.

Longer chatter on this topic is to test the full ISO Range of every camera as IMATEST can be off by about 3 stops there. If you take into account image processing and run some tests it gets even more interesting if that's your chosen workflow.

I mentioned in another thread, CineD's and my measurements will be different as I don't use IMATEST, nor do manufacturers when they create their ratings, but their general stop disparity between systems is in the realm of correct.

One thing I do underline is that the major differences now between these systems are pixel design, sensor design, imaging signal architecture, in camera image processing (or not), software side processing, and generally optical paths on a pure image quality front. There's certainly a conversation about actual RAW versus other methods on the codec side, but that's a deep rabbit hole now.

Though we are likely a couple or more years from it, Arri producing a larger format system with decent frame rates, resolution, and similar-ish architecture (likely a new sensor design) priced around or below $150K (thinking more like $100-$120K-ish by the time this happens) for the body is the actual competition for V-Raptor and even Monstro as well as the Venice 2. Canon has been very, very quite lately I suspect they will be dabbling with at least a 8K 60fps camera maybe next year.

I personally have a long, long list of features I want in whatever next gen cameras come out. I'll save that for off the air as of now.

 

[Jon Pais]

I'm confused. The review they gave is STUNNING. They basically say that the Raptor is incredibly close in performance to an Alexa Mini LF (for exposure), even though it costs what 3-4x less? And has way more FPS?

To even get close to an Alexa Mini LF is a staggering accomplishment for the price:

Our current full frame benchmark for dynamic range is the ARRI ALEXA Mini LF with 13.5 stops at SNR = 2 and 14.7 stops for SNR = 1, without highlight recovery. The ARRI ALEXA 35 (Super 35 sensor) exhibits 15.1 and 16.3 stops at SNR = 2 and 1 respectively (also without highlight recovery).

Judging from the waveform and IMATEST results, I would say the RED V-RAPTOR has about 1 stop more dynamic range than the best consumer full frame cameras. Whereas the ALEXA Mini LF has about 1 stop more and the ALEXA 35 three stops more dynamic range than the RED V-RAPTOR.

And you can't compare the Alexa 35 to ANY CAMERA right now, it's in a league of it's own - and costs as much.

RED is going back to what they've always done - for the $$$, outperform everyone and give indie producers a real advantage. But an Alexa 35 is still in a class of its own and that's OK.

+1 Nick. It looks like RED is killing it.

 

[Clayton Burkhart]

I think the least discussed and most pertinent element which Cined looks at is latitude. In fact I used to think that I liked the Alexa range of cameras for it's dynamic range, but in fact what I liked was the great flexibility in post that meant I could match images very well, even if they were exposed differently. It is this which provides the safeguards in many circumstances. It seems to me that not only does the Raptor provide excellent DR, but also I believe Cined found it had 9 stops of latitude, which is just behind the latitude of the Alexa universe. Really, that is quite excellent and I gotta be honest, I have always disliked RED cameras (I won't go into it) so I congratulate RED for staying relevant in such a competitive market.

 

[Mike P.]

Ironically, there are two different threads on RU about the same article; one excited about it and the other disappointed about it. It read very neutrally to me, showing how Raptor measured and how those metrics compare to other cameras measured the same way (with little commentary outside of that).

Everyone’s gotta admit though, it’s pretty wild that the 12+ year old ALEV3 sensor design has a stop more than last year’s Raptor, which was Red’s <checks notes> *seventh* major sensor design released in that same time period… MX, Dragon, Helium, Gemini, Monstro, Komodo, Raptor… And for every one of those red sensors the RU pundits always (falsely) claimed ‘It finally has as much as Alexa!’ In that regard, it’s nice to see even the most diehard redusers are finally willing to admit that ALEV3 has more latitude/usable DR.

Also like the discussion of what’s worse; internal noise reduction (that is seemingly no more destructive than a black shade done every frame) and internal highlight reconstruction (whose most substantial benefit is two additional b&w highlight stops that conveniently help chart numbers, but aren’t really unusable in saturated images).

 

[Björn Benckert]

Would be interesting to see them doing the same lab test with Monstro. Is that done? Could not find it on their page. My guess it would reach the same results in terms of stops. Have not played with Raptor, everyone says its cleaner than monstro but I doubt that monstro would not be able to pull 12+ stops on their chart.

but as I see it lab test stops is not everything. Usable stops means more and high resolution and downsampling means a lot when looking at what can actually be used in the dark end of the spectrum.

To me its a bit okward that they show and mention the extra highlight reconstructed stop that the reds come with trough processing but then they don't explain much what it actually do as it's quite a nice feature and most often extremes highlight are nothing but close to monochromatic and rarely you want to pull colors from such exposures but you still want a nice rolloff to pure white. In other words that extra stop is very useful but is kind of accounted for as nothing in their test.

But all in all it's, quite easy to see that much have not happened in terms of camera DR except for the Alexa s35 which is in a class of its own.

Doing lab test is one thing. But in comparison raptor is really a beast with its small size and high frame rates. We just wrapped a commercial shot on Venice and I still find it really odd that it does not do more than 48fps in full frame, same with the LF. From having my monstro for, what, 5 years or so, I find the frame rates as most limiting on the LF and Venice. Then the size, low power consumption, price and wide sensor format of the raptor is way more appealing to me than what the others are strutting. Venice got clean colors but I still find monstro better when shooting them side by side.

 

[Leonardo Newman]

I think the idea that they don’t count the highlight recovery stops but somehow counts the shadow stops of the Alexa 35 with it’s built in Noise Reduction really strange. In any event their own numbers for the Venice 2 contradicts their claim that you must subtract a stop from Raptor. Here’s the interesting thing with these charts. The Raptor registered “13 stops on the chart not counting highlight recovery”, but measured 13.7 SNR =2 and 15.1 SNR =1 in Imatest (but apparently that includes highlight recovery), Venice 2 chart registered at 13 stops, but measured 12.9 SNR =2 and 14.7 SNR =1, in imatest with no highlights recovery, but in the latitude section, their words are, “the Venice 2 measure 8/stops latitude in raw, and wiggle room towards 9stops latitude in proress“, their description of raptor was, it’s measure 9 stop latitude with wiggle toward 10 stops, but for the DR on the chart 13.7 SNR =2 and 15.1 SNR =1 they say we must subtract a stop because imatest isn’t able to discern the highlight recovery, meaning we should consider raptor as 12.7 SNR =2 and 14.1 SNR =1, this would put it behind the Venice 2, but cined own latitude test proves raptor to have 1 stop more than the Venice 2, which would then put it back to the 13.7 and 15.1. So this must mean that the stop they say is unusable because it is monochromatic must be completely useable since the latitude test that is all about useablity shows raptor to have a stop more than Venice 2. Meaning they need to count there Imatest results of raptor as useable putting raptor 2nd in Dr on their database only to the Alexa 35.

But maybe I’m missing something and thought about this too deeply.

 

[Mike P.]

But maybe I’m missing something….

It’s because whatever Arri is doing internally results in wholly usable, colour accurate/consistent stops of latitude, whereas the highlight reconstructed stops of RED are *at best* monochrome and at worst (the *second highest* stop) gross looking gaudy yellow (to the point where it'd almost be better if they made that one monochrome too, since it negatively affects what should otherwise be a saturated highlight). On top of that, CineD's examples demonstrate why noise reduction doesn't salvage any more with RED's low-end, as both ALEV4 *and* the 12+year old ALEV3 offer equivalently useable -6 to RED's -6. And those are attempting to be "scientific" (objective) results; the delta could be considered slightly larger if you're going by finished images of an actual scene.

The middle ~7 stops of all three brands -- RED, Sony, and ARRI -- are pretty much the same in their usability. It's the extremes where they comparatively fall apart on RED, even though they're still counted by IMATEST/RED specs. That you can roll-off the measured reconstructed highlights and crush out the measured colour casted shadows (aka reduce the visible DR) so they don't look quite as bad is the status quo for any camera, but with ARRI you could actually use more of those stops. Actually, it's worse than that, since REDs are more heavily weighted below-mid-grey.

 

[Leonardo Newman]

​​​This is not my experience. There is never any gaudy yellow before clipping with red. I've owned 3 reds, never saw this gaudy yellow you speak off. Additionally, those reconstructed stops do sometimes have color, I've posted examples below. Additionally, Cined and CVP's test shows raptor having the most useability as it relates to underexposure between all the other high-end cine cameras. Cined registers Raptor at -8, not -6. So not sure what you're saying as it relates to that. At the very least, raptor is on par with mini LF; if reconstruction is taken into effect, then it pushes the raptor over the edge against LF. That is what I understood from that test reading their charts, I disregarded their reporting because it did not line up with their own charts. In my experience and testing, the reconstruction is very much useful and also brings back color if you process the image correctly. When I originally shot these images below I was in the red FB group talking about why red desaturates as it goes to clip and wondering why I can’t get color in those final stops, and then I tested more, read alot of old post on this forum and learnt how to process my images in order to get that color up to clip. I then went back to those same images that I thought was completely grey and applied that new knowledge and all of a sudden I can recover almost a stop More. Now some people may say well why do you have to work that hard, what can’t it just be available, well you can always just expose at higher isos and there you go. I’ve done test at 6400 iso on raptor in daylight, no one would ever know, because the noise level is still so low, and just like that you get 3 extra stops of highlight protection and thats as simple as it gets. But I’m excited that in post I’m able to get even a little more, I don’t think that’s a bad thing. So I can shoot at 800 or 1600 and stay completely clean and then squeezing out that extra stop if need be in the Raw tab. I’m a colorist so I find it awesome to be able to do that.

filedata/fetch?filedataid=119692

​

​
Here's what most people think highlight reconstruction leaves you with as you approach clipping.
This is just a normal image that I intentionally overexposed and tried to bring back by dropping iso in resolve. Even in this their is no gaudy yellow that you speak of Mike, though there is the monochromatic behavior.


​
filedata/fetch?filedataid=119691

​

​
​Here is what highlight reconstruction can actually do if you properly process your image. While there is more than enough dr in Raptor that you should never have to rely on highlight reconstruction, if you find yourself needing that extra bit, this is what can be achieved because of highlight reconstruction. These are the exact same images. Notice how additional detail is brought back as well as color. So if you count built-in noise reduction in your dynamic range total then you should also count built-in highlight recovery because they are both equally useful.

 

[Zack Birlew]

I always find these tests interesting but ultimately not very helpful when going to set. Really, proper exposure techniques net you 99% of your work and if you're fooling around with deep camera settings and lighting setups for every single shot in order to maximize dynamic range then that must be a miserable process. Knowing that you can recover highlights and shadows is good to know but when you're designing your shots, the goal is proper exposure regardless of what can be stretched, crushed, or pulled in post. Working on the color and final VFX on my first feature right now, I can tell that going with a RED One MX was a great choice as it was a major leap over other options in terms of stretching the image in post, especially adding layers of exposure windows for effect or to balance out the image in some of our time constrained production moments. Still, there were some expectations and limits with post-exposure adjustments that I'm sure newer cameras could edge out that extra bit but those just weren't an option when we filmed.

Things that would really affect the image would be LUTs and, in this case, what about throwing on an Arri Alexa LUT? How close can that get you to Arri's dynamic range and overall look? Would that make the whole process easier to get a high dynamic range image or would it be about the same? If a LUT is the answer, then what's wrong with RED's color choices? How can they be made better or at least more accurate?

As it is, I'm not sure what extra dynamic range beyond what we have now would do as I saw in a recent HDR show, I just don't think I needed to delineate the gradations of light to the white tube of the neon signs on the wall in the background or that I absolutely needed to see the intricate lace pattern of the curtains against the bright window. As we all know, there is a point to letting some highlight details bloom out or letting them completely blow out if that's how the shot needs to be for the subject to be properly exposed. If there's one thing that would help cameras right now it would be lighting as we keep getting nice robust LED lights that are bright but we need to get past flood lights and get some more inexpensive high wattage LED spot and fresnel options to hopefully help break out of this flat lighting phase everything seems to be going through. What good is dynamic range if everything is so dimly lit and left in the shadows for the most part?

 

[Björn Benckert]

I shot monstro and LF and Venice side by side I don't know how many times and yes I run a VFX shop / is doing flame work on a dalily basis since 23 years back and on some instances I composited both Alexa, monstro and Venice material into the same very shots, shots that won Clio awards.

If all these three cameras are exposed about a stop under clip for the highlights then Monstro wins every time. And that does not mean all cameras with the same stop on the lens as they all clip differently you need to find the clipping point for the sensor and then pull a stop back. If doing so, and bringing them all in a linear color workflow monstro 8k simply delivers the cleanest and most elastic image with more usable lowlights of the three. Sony is normally more color accurate out of the gate, but that's nothing that arri or red can't balance out.

Again I have not tested raptor but people say it go cleaner black than monstro. Then regarding the highlight reconstruction thing its actually something that can be done with any image in post red just implemented such code in their debayer which is clever and make sense, Im actually surprised that others have choosen not to do so as it extend the DR and in most cases in a quite accurate way.

 

[Leonardo Newman]

Zack, as it refers to luts, using Arri's lut isn't going to give you any more dynamic range than Red's lut. I've tested this. What's happening is sensor-level design. Arri with its dual gain architecture weights its DR toward highlights straight out of the box. Red weights their DR toward shadows straight out of the box. Even further, both the 709 luts are going to squeeze DR so you'll end up losing dr on both once the lut is applied. So you always want to do your grading before those luts. Additionally, changing from log3g10 to Logc3 or LogC4 will not give you any more dynamic range either. Its all about the information that you capture. And that takes us to B'jorn's point.

These tests always tend to favor arri because internet people seem to not count shadows stops as dynamic range, we only care about highlights. But after watching some older movies I see why a lot of bigger DP's even Arri's own Art Adams say that they prefer a camera that can see better into the shadows. If more internet people cared about shadows then a lot more people would realize how Red really might have more useful DR. It's much easier to underexpose slightly and light a face than it is to light a night exterior. Now shadows, and highlights, everyone has their preference but watching these older movies shot on film that have virtually no shadow detail shows why DP's really wanted better shadows.

I think how B'jorn tested those 3 cameras are how we should do all test. Test each camera equivalently not equally. Equally says let's set all cameras to the same settings and see who wins. Doing this method will always place some cameras at a disadvantage based on how DR is weighted at the sensor level. Equivalence says let's test each camera in the same way, therefore let's take each camera to a stop below its clipping point, which is different for each camera, and then see which camera has the most useable image, and as he says Red wins because it has less noise and more information. This is a much better way of testing because you are respecting each camera's sensor design as it relates to each specific sensor. This takes all variables out of the equation such as ISO as its relates to Raw. Or in other words, you are testing each camera based on how the manufacturer designed it to be used and not just trying to use every camera the same. Phil is always saying we should test cameras at ISO's where there is the same noise level for each camera, i.e. Arri at 800 has the same noise level as Monstro at about 2500. This again would be testing each camera equivalently not equally. But this would clearly show useable DR to at least tied or in favor of red.

I never worked with film, but I've read constantly about how you had to treat each film stock differently, whether lighting for 500t or 100t, kodak, Ektachrome, fujifilm etc, so why are we now trying to treat every camera the same when each manufacture designs their camera in different ways?

I am fully persuaded that maybe outside of Alexa 35, no other cameras has more useable DR than Raptor, now is it 17 stops of DR? I don't know and don't care. I just know it doesn't get much better than Raptor.

 

[Björn Benckert]

I always find these tests interesting but ultimately not very helpful when going to set. Really, proper exposure techniques net you 99% of your work and if you're fooling around with deep camera settings and lighting setups for every single shot in order to maximize dynamic range then that must be a miserable process. Knowing that you can recover highlights and shadows is good to know but when you're designing your shots, the goal is proper exposure regardless of what can be stretched, crushed, or pulled in post. Working on the color and final VFX on my first feature right now, I can tell that going with a RED One MX was a great choice as it was a major leap over other options in terms of stretching the image in post, especially adding layers of exposure windows for effect or to balance out the image in some of our time constrained production moments. Still, there were some expectations and limits with post-exposure adjustments that I'm sure newer cameras could edge out that extra bit but those just weren't an option when we filmed.

Things that would really affect the image would be LUTs and, in this case, what about throwing on an Arri Alexa LUT? How close can that get you to Arri's dynamic range and overall look? Would that make the whole process easier to get a high dynamic range image or would it be about the same? If a LUT is the answer, then what's wrong with RED's color choices? How can they be made better or at least more accurate?

As it is, I'm not sure what extra dynamic range beyond what we have now would do as I saw in a recent HDR show, I just don't think I needed to delineate the gradations of light to the white tube of the neon signs on the wall in the background or that I absolutely needed to see the intricate lace pattern of the curtains against the bright window. As we all know, there is a point to letting some highlight details bloom out or letting them completely blow out if that's how the shot needs to be for the subject to be properly exposed. If there's one thing that would help cameras right now it would be lighting as we keep getting nice robust LED lights that are bright but we need to get past flood lights and get some more inexpensive high wattage LED spot and fresnel options to hopefully help break out of this flat lighting phase everything seems to be going through. What good is dynamic range if everything is so dimly lit and left in the shadows for the most part?

Luts does not give you more DR. Think of DR as the size of you bat to hit the ball with and luts as a tool to steer the ball after it hit the bat. You MX is a thin little stick compare to the Alexa that is more like a big tree trunk in comparison. The Mx highlights burn / goes toast earlier and in the shadowend you hit the noise floor earlier. A lut can't change anything if you did not hit the ball so to speak.