New Epic-W vs. Alexa Mini test

[William Tracy Babcock]

This test does have a big impact on RED Helium owners like myself. This test was showcased on nofilmschool which is, for better or worse, a high-traffic site where people in all levels of film production go to get some of the latest info on all things cinema related. As it stands, we all know that the Alexa is winning the high end cinema an TV camera war. Just take a look through IMDB. RED is still used on many productions that use digital cameras but it is now in the minority when it was once the standard (since it pretty much invented larger sensor 4k digital cinematography). This test effects owner/operators like myself because it reaffirms the film industry's bias towards the Alexa which means it will be a harder sell to my clients who already want to rent the Alexa vs my RED cameras. I already have a friend, a fellow RED owner, who is not upgrading his camera because of this test. He's sick of fighting with production. He told me that if they want the Alexa so much I might as well just leave the RED pipeline and get what the film industry is already using as a standard.

Having the camera rated at 800 iso vs 1600 iso for this test is a big deal. That would have at least dropped the highlight retention gap by a full stop. But I'm saving most of my ire, in this situation, for RED. Why did they release a camera without fully testing and refining their color science? They definitely rushed this camera because they know something big from their competitors is on the horizon. RED needs to roll out whatever they have up their sleeve to improve the Helium sensor ASAP in order to show once and for all that this camera is great and a worthy competitor to the Alexa.

I downloaded the RAW files from the test and used the RED to Alexa Truecolor LUT and definitely had a much easier time matching the shots between the two cameras (minus of course the parts of the image that we're overexposed) vs the original IPP2 LUT. Also, using the Truecolor LUT also gave me much more love in the shadows than is showcased in this test vs the original IPP2 Transform. RED needs to step up its color science game fast. Why can't they finally admit that if the film industry is leaning towards a certain image look why not beat them at their own game? Sony, Canon, and Panasonic have finally realized this and their own transforms that mimic the Alexa LUT are hugely popular. I know many a Canon C300 MKII and Sony F55 owners who are so grateful that their brands finally so the writing on the wall and built image transforms that get them closer to the Alexa look/color. I also think they should come up with a new OPLF that is specifically tuned for the Helium camera vs the other way around which is tuning the Helium sensor to older OPLF's.

I've recently been blown away by how much this sensor can resolve with good lenses. I just started shooting all kinds of stuff with my Leica R's at F22. I've never seen any other camera resolve detail like it. Now if they could only give me a softer highlight rolloff while keeping all the shadow detail it currently has with better color nuance/contrast (especially deeper cyan/blues) it really would be something special.

Unfortunately I have to agree 100% with this sentiment. First a little pre-amble...

I bought an Epic-M Dragon in 2014 as it was the first camera I saw that could compete with the Alexa. It wasn't 100% there with image, but was considerably more affordable, and held distinct advantages in size, weight and RAW capability. The camera worked great for me over that first year and then come NAB 2015, the Weapon & VistaVision were announced. The Weapon body upgrade was too expensive to justify in my opinion, but coupled with the large-format VV sensor it made sense. I purchased both, but only because it meant upgrading to the larger sensor, which was supposed to ship by the end of that year. Regardless of the other facets of the camera, the large-format capability was something that I not only personally coveted, but would really stand out against the Alexa line and be a boon to my rental business. It's something I could easily sell as a differentiator in an extremely crowded and competitive market.

It's now coming up on 2 years since I put that money down and still no VV in my hands, nor it seems anytime soon. During that time, the Mini released and took a large percentage of the rental market back from RED (at least in NYC) as it could now easily compete in the size and weight department, yet with superior DR and a workflow that Producers still love. I also received my CF Weapon upgrade, which while great from a usability standpoint, only marginally improved the image, which stands paramount. A solid upgrade, but again, not one I would have paid for if not for VV. This upgrade also took longer than initially announced.

A scant 3 months after I received my CF Weapon, the Helium was stealth announced/released. At the beginning there was very little info, and as a customer who'd put a deposit on a camera announced 16 months earlier, this was extremely frustrating. A couple months later came the official release along with the announcement that us VV depositors would be receiving a Helium as a placeholder. I was pretty miffed about the VV delay, but understood that there were manufacturing issues and this seemed like a decent short term compromise. I also thought, hell, seeing as this is the newest 2016 tech sensor it might end up being so superior that I end up not sweating the VV delay.

That brings us to today. This test along with many others seem to show that the Helium is being beat out in many facets by a sensor released in 2010 (Alexa). In fact most tests I've seen other than DxO even seem to show the 4 year old Dragon sensor having superior DR. This brand new sensor shouldn't be stuggling to measure up, but instead wiping the floor with 7 year old tech. This is a terrible look for my rental business, and even worse, on jobs I'm DP'ing it makes it hard to fight producers who favor the Alexa.

I understand IPP2 is unfinished, but the camera has been available to the public since October. I'm hoping there's more to be mined from this new sensor, but nothing yet stated or announced on RU has shown me that to be the case in any significant way.

I'm taking delivery of my Helium Weapon on Monday, and plan to do immediate extensive testing myself. I sincerely hope this proves some of the measurables in these tests wrong.

And before all the responses to the effect of "a camera is a just a tool". I know. I 110% agree. I've made many images I'm extremely proud of with my RED cameras, and I have no doubt the Helium will be the same. That being said, these cameras aren't cheap and have all required loans on my part. Loans which require me to have a rental business to pay them. And if a camera released in 2016 can't outperform a sensor released in 2010 that makes the rental business pretty tough. Just as Anton has, I've known a few people who have or who are jumping ship for this reason. I really don't want to be one of them as I'm heavily invested in the RED ecosystem and am still eagerly anticipating the VV.

Seeing as RED is using that DxO test prominently in marketing on their website, I really hope the real world performace is soon able to match it.

 

[David Mullen ASC]

The area where the Alexa always seems to beat the competition (not just Red, but everyone) is not WHEN things burn out to white, but HOW they do it. The "when" can be controlled by exposure or ASA rating, but the "how" is outside of what the cinematographer is doing. All the other areas, from shadow detail, resolution, noise, low-light capability, dynamic range, etc. there are pluses and minuses with every camera and a lot of it comes down to taste and the needs of the particular project. But I don't know why the Alexa seems so particularly good about extremely overexposed skintones burning out to white in a natural, non-weird way color-wise that seems more "film-like". Sorry, but that's the one test that the Alexa always wins, year after year.

Beyond that, obviously there are things that the Helium does that the Alexa sensor can't do.

Jared is right that perhaps there needs to be more education out there, but it's a valid aspect of testing to see how a camera behaves if used in non-optimal ways since we all end up shooting scenes in non-optimal conditions. Of course, in a comparison test you want to be "fair" but it's not invalid to compare how two cameras behave if the same ASA rating is used, and one should be able to extrapolate what would happen if you used a slower or faster rating. Obviously if one camera clips at a point that is one-stop earlier than another camera, you can guess (or better yet, test) to see if rating that camera one-stop faster would match the clip point and then compare the other end where noise becomes an issue.

As for waiting to test cameras until all the software and hardware upgrades and processing programs are optimal, well, that's never going to happen if the camera is on the market. Once the camera is out there in the real world, you have to expect footage and tests to start to get released, at all sorts of quality levels.

 

[Noah Yuan-Vogel]

I think the takeaway here is that ALEV III is still pretty much on top for color and DR characteristics, but by a very narrow margin, and that doesn't account for noise/sensitivity and resolution benefits from Helium. Frankly I'm not surprised by this, there is a reason ARRI is still using the same sensor in their 3rd and 4th generation Alexa cameras after 7+ years and those cameras still cost more than RED's.

I think up to and including Epic Dragon, most understood that going RED over Arri represented sacrificing some of those image characteristics I mentioned for the resolution and flexibility of RED but that those are real benefits and sometimes requirements. We can pretend to be surprised that DSMC2 Dragon narrowed but didn't fill that margin and that Helium introduced other benefits and tradeoffs, but it is only through willful ignorance that one would say any RED camera completely supersedes Arri's offerings. It is still big news that RED offers greater flexibility and resolution at lower costs with fewer tradeoffs than ever. It's important to test with many different methods since these cameras have many uses and their users have many approaches and there are a lot of complexities to these cameras that may be hard to quantify otherwise.

It will always work to RED's disadvantage that they have different sensors and different bodies and different pipelines that will trip some people up, vs alexa/amira which maintains fairly similar performance through the whole line over many years. Hopefully RED will make up for that at least somewhat in its greater accessibility and strong community and educational outreach, but expecting anyone to have a perfect handle on how to best use a camera lineup that is constantly changing both in sensors and OLPFs and firmwares is often unrealistic.

 

[Rakesh Malik]

Frankly I'm not surprised by this, there is a reason ARRI is still using the same sensor in their 3rd and 4th generation Alexa cameras after 7+ years and those cameras still cost more than RED's.

It's an amazing bit of engineering, for that sensor to have lasted this long in spite of the pace of technological progress!

 

[Steven Shea]

I think the takeaway here is that ALEV is still pretty much on top for color and DR characteristics, but by a very narrow margin, and that doesn't account for noise/sensitivity and resolution benefits from Helium. Frankly I'm not surprised by this, there is a reason ARRI is still using the same sensor in their 3rd and 4th generation Alexa cameras after 7+ years and those cameras still cost more than RED's.

DR by a narrow margin, but I think many see the Alexa as having notably superior color science.

 

[Noah Yuan-Vogel]

Well... I'm not so sure. Many photographers prefer the Leica M9's CCD to almost any modern CMOS, and yet it scored only 69 points. However, DxO measured the Helium at 15 stops of DR. The Alexa does not have 15 stops, the last time I checked. Please correct me if I'm wrong. Also, it has a built-in low-con filter. The RED does not.

Depends how you measure. Certainly Alexa can see 16+ steps on a Xyla chart...

 

[Noah Yuan-Vogel]

DR by a narrow margin, but I think many see the Alexa as having notably superior color science.

yes, so it seems, especially in the shadows.

 

[Nick Vera]

No, they shouldn't, because that's not how you'd use them on set. You'd base your settings and exposures on how the camera you're using works, not on how the other camera you used on that other shoot works.

What they did was akin to taking some film stock rated at ISO 500 but exposing at ISO 250 because ISO 250 is what they're used to.

If you're doing a direct comparison of each other you CAN keep them at their native ISO, but you have to bare in mind that one is going to clip in the highlights sooner than the other due to two different ISOs. I'd rather see a test where the Helium is at 1250 or 1600 or whichever ISO you prefer and have the Alexa at the exact same ISO, and then shoot them at lower ISOs. I've already read the film quote before and unfortunately ISO doesn't work exactly like film. There are ideal ISO's in the different cameras that shadow and highlight ranges are evenly distributed. Where as you if push or pull film, you're going to lose stops.

 

[Nick Vera]

Mr. Mullen said it best

 

[Tim Daust]

They seem to hate us "redusers" a lot for voicing what we see wrong with this test. They claim we are the ones who are "defensive" on the "details" of small technicalities..

Redusers - while most of us are opinionated and defensive about our ecosystem, I find that we all typically don't "hate" ARRI.. most red owners I know are surprisingly camera agnostic even though "they" say otherwise about us when we complain about the misrepresentation in this test. meanwhile most "anti red guys" religiously support ARRI. The only real way to do an unbiased test is for the people performing said test to not have any prior bias towards RED - bottom line. (Knowing the specific cameras "technicalities" are also preferred such as base ISO)

A similarity among the comments comparing RED cameras to ARRI is that "anyone" can get the same great results from any ARRI straight out of the box without having to spend hours on reduser educating themselves how to get the best results from RED. When I was die hard Sony user, I spent weeks (years even) discovering how to get better images out of my cameras.. out of the box it was a "video" camera but in the hands of the right user transformed into a powerful digital cinema tool..

The only thing I expect to work perfect out of the box are my tennis shoes.

Our level of knowledge/experience/creativity combined are what separate our bodies of work as a cinematographers right? If you expect to be able to produce perfect images "out of the box" with any camera this seems unrealistic to me. A reasonable expectation of producing "acceptable" images out of the box is all you should have when purchasing any camera no matter the price tag. Our individual understanding to capture light and create images that support the narrative/creative always wins. Where would the room for advancement be if we all expected the same equal performance out of any camera regardless of personal experience level. Where would our bodies of work be??? That experience directly correlates with the tools we use.

It is only a matter of time before IPP2 becomes more widely adopted and hopefully the gap between RED & ARRI is closed. Exciting times indeed.

I'm sorry for the long winded rant.

 

[Ben Scott]

I don't think they are camera agnostic at all.

We/they seem desperate to prove we're as good as those other kids rather than just accepting that the thing the Arri is best at, happens to be one of the things most DOPs look for in an image. That rolloff David mentioned earlier.

The things Red is best at are slightly different and completely valid - amazing resolution and sharpness and great post workflow for people who like to dig in. And there's a lot of things on a par with, but different.

But let's not lie to ourselves here - If the Alexa Mini came down to the same price as the Scarlet-W a majority of Scarlet-buying DOPs would buy the Arri. If the Alexa 65 was able to be purchased at Weapon VV prices....?

EDIT: Of course then the flipside would be you'd then see loads of terrible Alexa Mini videos in the hands of people who don't know how to get the best out of them too

 

[Aharon Rothschild]

This test is important and is the sort of thing a DP would do before a feature when shooting on film. Going off the charts and claims here I would show up on set looking for 16 stops from the Helium sensor and blowing out at 13. The point is not which camera is better; consider this a free lighting and stress test and as such we should welcome this very useful information.
There is some discussion here about the dynamic range of the two cameras matching and that really won't help on set. The .r3ds of the xyla chart posted earlier match up with the results of this test. Again we shoot a film with the camera we have so there is little point in discussing a winner and a good amount of useful discussion in how best to meter and expose for Helium.

 

[Mark L. Pederson]

I will say two things here -

#1: What David Mullen said. Everything he said.

#2: With respect to ISO when comparing RED to ARRI - you need to FIRST understand that the two camera manufacturers do NOT use the same math for their ISO scales.

In other words ... ARRI's ISO 800 on camera does not equal RED's ISO 800 on camera. If you put the two cameras side by side - same output gamma - same monitor - same aperture and shutter - you will NOT see the same exposure on the monitor. I can't remember if it 's just under or over one stop - but not an insignificant. difference.

When I tell this to people - they kinda freak out and say "you mean I can't trust my light meter when shooting RED"? RED uses the same math as Kodak used for their ISO scales - not sure what math ARRI is using - but I can tell you for sure - not the same math. It could possibly be that there are a couple different methods for calculating - digital still, etc. - so .. you can argue they are both correct - butt different.

But just like when Kodak or Fuji put out a new film stock - most folks would shoot their own bracketed test with their own exposure methods - and the decide how to rate the stock.

I told several people at RED that they should have the Helium default to ISO 1280 - if for no other reason - than to remind them it's NOT a Dragon sensor (or an Alexa sensor for that matter) - and one should not just "set to 800 and go" if you want hte best results from the camera.

 

[Nick Morrison]

I will say two things here -

#2: With respect to ISO when comparing RED to ARRI - you need to FIRST understand that the two camera manufacturers do NOT use the same math for their ISO scales.

In other words ... ARRI's ISO 800 on camera does not equal RED's ISO 800 on camera. If you put the two cameras side by side - same output gamma - same monitor - same aperture and shutter - you will NOT see the same exposure on the monitor. I can't remember if it 's just under or over one stop - but not an insignificant. difference.

When I tell this to people - they kinda freak out and say "you mean I can't trust my light meter when shooting RED"? RED uses the same math as Kodak used for their ISO scales - not sure what math ARRI is using - but I can tell you for sure - not the same math. It could possibly be that there are a couple different methods for calculating - digital still, etc. - so .. you can argue they are both correct - butt different.

But just like when Kodak or Fuji put out a new film stock - most folks would shoot their own bracketed test with their own exposure methods - and the decide how to rate the stock.

This is a great post Mark, and something my team and I talk a lot about. To your point, our Sekonic light meter doesn't appear to match our Epic Dragons exactly. In general we light for 320, even when we're setting the camera at 800. At the very least, we always try and give the camera an extra stop. And the final arbiter of exposure is always the histogram, when previewed in RAW. If the histogram is fat, and nothing is clipping, we know we're good.

But in terms of connecting our light meter to our camera, yeah...we've had to essentially make our own cheats to connect them together.

Thanks for bringing this up - it's a subtle topic that isn't always discussed.

If you know more, or can expand more, on RED's evaluation for ISO, would be really helpful. Maybe there's a way to set the Sekonic to a similar "scale"?

best

 

[John Marchant]

Re: using a meter - Sekonic provide test targets and software for calibration. You really should calibrate the meter to any camera you want to work with

 

[Martin Stevens]

I thought there was only one correct ISO? Does Red use it, or Arri?
Because apparently there has been a historical difference between
them in regards to ISO.

 

[Karim D. Ghantous]

Re: using a meter - Sekonic provide test targets and software for calibration. You really should calibrate the meter to any camera you want to work with

I didn't know that, but it kind of makes sense.

I remember reading in photo magazines how some people preferred to rate Kodachrome 64 at ISO 80 to increase saturation. While some rated Velvia at ISO 40 because it had too much saturation. But those are slightly different things.

 

[Aaron Lochert]

Just wanted to share my (subjective) grade/test with their footage. I'm sure a competent colorist can do a better job.

More of a a real-world example for a 4K/UHD delivery. If we're calling the ISO 800 Alexa image the benchmark for highlight rolloff-to-noise ratio, it seems fairly easy to get there with Helium:

Alexa ISO 800 T2, no LUT to hide clipping point:

Helium ISO 1600 T2.8, no LUT to hide clipping point:

So now you can see the clipping points are much closer. Alexa still has the edge by a third, maybe half stop. Alexa also clips only the red channel in the lamp, so the upper left corner goes yellow. The Helium is clipping all three channels in the corner of the lamp, making it go full white.

Alexa ISO 800 T2, LogC to Rec709 LUT:

Helium ISO 1600 T2.8, IPP2 low-contrast, rolloff-4 LUT, 5-minute color/contrast adjustments:

Noise levels are much closer together now, too:

1:1 Scale UHD, graded (don't judge sharpness here, looks like the Epic-W shot has buzzed focus):

 

[Björn Benckert]

Again, I find it importsnt to point out. No cmos cameras have rolloff. they all fill up their photosites linerearly until they clip. All cameras from red behave this way and also all digital cameras from arri behave this way.

So saying on camera has more or smoother rolloff than the other does not make sense.

Luts and grading can create a dynamic roundning towards white cameras can not do this. Atleast not if shooting raw.

If you stick a ND.6 in front of your helium you will see it will have more highlight protection / possibility for rolloff than the alexas and still with the filter in place you will be able to dig out more detail in the shadows from helium compared to the alexa shoting same settings but without nd.... To me that clearly states what camera has the upper hand. But sure alexas can just as red or most other cameras today create good looking images if the are set up right.

So with that in mind I find it easier to skip the whole iso discussion and instead try to messure cliping points and sensor floors as to me thats what matters the rest is just irelevant numbers, no?

 

[Mike Quill]

I have been frustrated with Red since they rolled out all of their Weapon offerings, and then the Epic-W. Why announce/release so many confusing camera iterations and sensors within a one year period? Not to mention Red's declarations that didn't quite hold up: "if you don't upgrade by such and such a date your camera is dead end" and "if you don't upgrade to this body/sensor your camera will be dead end."