ASC and BSC camera tests ...

[Stephen Williams]

I'd still like to know what version was used. One of the difficult things to get across to some people is that, whatever it was like on version 16, or 13 or 17, has little bearing on what it's like under 20. A lot of blanket statements about the camera hark back to older firmware versions and are so no longer relevant.

One of the dangers of releasing a camera before it was ready IMO. BTW I am talking about V8 & below

 

[David Mullen ASC]

The ASC test was done in January and I think the BSC test was done in February or March, 2009, and they would have used whatever the latest builds were at that time.

 

[Sanjin Jukic]

From my perspective according to info I have multi-camera tests from ASC and BSC both are irrelevant and obsolete to RED.

RED is and also always was a sort of FLUXUS that means constantly is changing towards a better quality.

 

[Stephen Williams]

From my perspective according to info I have multi-camera tests from ASC and BSC both are irrelevant and obsolete to RED.

RED is and also always was a sort of FLUXUS that means constantly is changing towards a better quality.

You make an excellent argument to shoot on film!

 

[Michael Brennan]

BSC test was not designed to get the best digital image on screen.

BSC test was not designed to get the best digital image on screen.

Here we go again.
The DP who lit the closeup stated that he lit the scene as he would film.
(Where have I heard that before!)

So I suggest the closeup test is a valid comparison of how a digital camera behaves if it is lit for film ie lighting to a contrast range that is beyond the range of the sensor!


To add insult to injury grading of the image for the close-up scene was not allowed...hardly the way to get the best image from a digital camera!
I don't know if the rational for exposure of the digital cameras also had a film based premise.

What the BSC have not done in this test and the previous is to shoot scenes where a DP lights and grades to the strengths of the particular format.

Such a test was done and transferred to film by Panavision where the some of the most experienced DPs in the world couldn't tell film and digital (Genesis) apart at the first sitting.

No doubt there will be another test in a few years and guess what! film will still have better contrast and skin tone out of the box, yet audiences will be paying good money to watch movies shot on digital.

In the light of this, research to explore the audience reaction to any differences in the aesthetic of film versus digital would be a welcome and forward looking initiative, rather than repeating tests that tend to divide the community.



Mike Brennan

 

[David Mullen ASC]

It's a completely valid test to point both film and multiple digital cameras at the same subject and see the differences. Personally, I don't know many people who aren't curious about such things.

There are plenty of demo films with gorgeous digital photography from Red, Sony, Panavision, etc. to look at if that's what you want to see.

Either type of presentation carries information with it that has to be weighed against other considerations and factors.

I'm sure it's true that the Red footage in the ASC and BSC tests could have been shot and processed to look better. But it's also true that the fact that there were some minor issues with the Red in some situations is also informative.

Besides, that close-up was so evenly lit in the BSC test that it hardly qualifies as being unfair to digital!

And a test where the lighting is kept within a safe contrast range for digital is hardly informative at all. How many of you would go out and spend the time to shoot such a test? If I could do only ONE test of a camera and/or film stock, it would be to see their failure points because I'd expect any of them to handle something easy and safe.

It makes complete sense to shoot a lighting test where the contrast range is excessively high to see that point at which each camera craps out either in the shadows or the highlights. I don't understand people who object to these sorts of tests -- are they expecting to never encounter high contrast situations in their line of work? Do they shoot everything on a stage where the lighting and art direction are always optimized for digital cameras? How lucky for them...

 

[Sanjin Jukic]

Obviously and commercially that Kodak kept way towards development of "film stocks' for movie images

but didn't happened to KEEP it towards the latest digital development in still photography (about making sensors were Leica still has illusion about it).

That's made a huge confusion HERE.

Acquisition with moving digital images is still going on (pretty & strongly well) but in another case Kodak couldn't deliver a chip...

THEN OTHER MAKERS TOOK OVER THE MARKET:

RED.

 

[Clint Johnson]

I'm not going to get my knickers in a knot over this but it does seem a little disingenuous to send the film stock off to a developer who will fine tune and tweak the processing to best take advantage of a specific film stock and how it was exposed while at the same time restricting the digital material to a generic and limiting process. Why do they think that all the pushing and pulling that takes place in the lab is okay while digital gets no love on th computer?

Or maybe I got it wrong and they did the same with the film stock? Maybe they would run Kodak 5222 and Fuji 8572 through the exact same chemical solution for the exact same time... just to keep it on a level playing field, right?

I'm all for setting up a scene that will show the failure points of the different mediums, that is the very point of a test like this, I would just prefer that they tried to get the most out of every participant rather than tilt the field toward a favourite team. If you want a truly level playing field, send the material off to a company that has a good track record with material from that camera, film stock or R3D don't matter, and tell them to process for exposure only- no colour correction allowed.

I'm sure they didn't drop their film off at the nearest one-hour photo lab* and I'd think that getting a company like Offhollywood would probably have given them a more accurate representation of what Red can do.

*are there any one-hour photo labs left in the wild?

 

[Gavin Greenwalt]

Film has been engineered to not need grading. Kodak has invested a lot of money to ensure the out of box result looks great.

I wouldn't point the finger at Kodak for having an unfair advantage. I would work with RED to encourage them to keep refining their out of box output. Total flexibility is great and I wouldn't want that removed but at the same time a default starting point like what I get back with film even from an automated machine tends to be to me a more appealing starting point.

 

[Shane Betts]

You make an excellent argument to shoot on film!

...but on which stock? The point many have to get their heads around (not talking about people here, just some of the people I come up against day-to-day, post supes etc.) is that a new build on Red is similar to a new stock from Kodak. Shove 1970's filmstock in a modern Arri and you'll get 1970's images (I know, lenses, but they are camera independent). Everyone gets that and we only test the newest stock. Red have designed a digital camera that offers the same kind of (hopefully) constantly open ended on-going development just as if new stocks were being developed but some still make generalisations based on build 13 etc., which is an error. Problem is, this camera is developing so much faster than filmstocks ever have and that seems to make some people suspicious.

 

[I Bloom]

I've shot my own tests with the new Kodak Vision 3 500T. I believe it to be better than pretty much all the digital acquisition formats and I wish more producers would shoot with it. Hell I'm wearing my Vision 3 cap right at this moment.

That being said, Stephen and David... Come on... this test is really ridiculous. They should put a Macbeth chart in the image and grade everything so that the charts match as best as possible. Then we can actually get a sense of the real difference between these formats.

All formats have specific gamma and color characteristics that must be compensated for to some degree. After that compensation the image will then retain certain characteristics that are difficult to compensate for, among them: dynamic range, detail, skin tonality, grain and highlight detail. In the age of DI this is the difference between acquisition formats.

This isn't just a digital thing. I've seen film DI's where the scanner wasn't properly calibrated to the stock and there was tones of highlight detail missing that you could see just by looking at the negative. Really beautiful stuff shot on reversal stock can fall prey to this problem.

Putting up an image from the F-35 and saying it's green is as ridiculous as putting up an image from Vision 3 500T and saying it's magenta. It doesn't take a master colorist to make them match, it takes a very small adjustment from a competent colorist.

 

[David Mullen ASC]

I'm not going to get my knickers in a knot over this but it does seem a little disingenuous to send the film stock off to a developer who will fine tune and tweak the processing to best take advantage of a specific film stock and how it was exposed while at the same time restricting the digital material to a generic and limiting process. Why do they think that all the pushing and pulling that takes place in the lab is okay while digital gets no love on th computer?

Or maybe I got it wrong and they did the same with the film stock? Maybe they would run Kodak 5222 and Fuji 8572 through the exact same chemical solution for the exact same time... just to keep it on a level playing field, right?

I'm not sure what you're talking about -- ECN2 color neg processing is pretty standardized and automated, there's no "tweaking the processing" unless you request something like push or pull processing, or skipping the bleach step. All color negative stocks, all speeds, Fuji and Kodak, go through it, though 5222 wouldn't since that's a b&w negative stock. And if the labs are following Kodak standards, the results should be the same at any lab. The whole color neg processing system is designed to NOT need fine-tuning and tweaking for individual rolls - in fact, it is near impossible to do so.

Why do you think they did something "special" for the film stock other than a standardized D.I. workflow?

--

The ASC tests were timed to match each other as closely as possible and I would assume the BSC tests were too. Now some would object to that, making the images match each other, but if you don't do that, then people get the wrong impression -- which is the problem with sending the footage off to different D.I. facilities that specialize in just Red or SI2K footage, etc. If you do that, then it will not match footage from other cameras and no one watching the results could get past that.

 

[Kevin Liu]

Is there anywhere online to see the ASC/BSC tests, or were they only made available to members? Thanks!

 

[Michael Brennan]

Besides, that close-up was so evenly lit in the BSC test that it hardly qualifies as being unfair to digital!

..

David, the commentary during the presentation drew attention to the hot highlights of the curtain and skin tone colour portrayed by the digital cameras.
The DP said he lit it for film, not for full high contrast sunlight!

I have no issue that most DPs and techhies can figure out that the images from the digital cameras can be vastly improved and that showing that film is a few stops better in the highlights is hardly big news.
But the audience for this presentation is also producers and directors who I believe could be turned off digital by looking at so many ghastly looking images where as most of the film examples looked good.
The cause for concern is that as yet there are no comparative graded results.

The same thing happened on the last BSC test which they took on a road show and invited DPs, producers and directors to attend.


Mike Brennan

 

[Neil W. Smith]

Is there anywhere online to see the ASC/BSC tests, or were they only made available to members? Thanks!

I don't think that the original test images are available to the public. I saw the ASC test at the PGA event on the SONY Lot in Culver City and then at the DGA presentation .... both theaters are high end DCP (Digital Cinema Package) and film projection environments ... the Arri film camera looked best followed by the Sony F35 ... unfortunately, the RED looked the worse on many of the tests ... the Panasonic 3700 looked better in most of the tests.

Even if you had access to the DPX files and the DCP package, you'd need to see the finished images in calibrated environment to make any informed assessment of the image quality of the 8 cameras tested .... funnily enough, at the DGA event, after they showed the DCP version of the test they screened a film-out version from the DPX files .... in my mind (and bear in mind I'm only a DI Supervisor not a DP) the Arri film camera didn't look as good and all the digital cameras looked better.

The concern I have with the tests results is this:

1) the lighting of the different set-ups and the post-pipeline used were specifically geared to the requirements of a 35mm film, DPX based workflow .... all the digital cameras had to follow the same lighting set-up and 10 bit log DPX finishing path. David Mullen has explained most eloquently why the ASC choose this as the benchmark and there is certainly a valid point to doing the tests this way.

2) I have suggested to several members of the ASC Technical committee that they could have taken two of the set-ups (an Exterior Day and Interior Night) and let the camera DPs adjust the lighting/filtration to take into account the response curve of the different cameras and then given the raw files to a post-house that has worked with the camera files to grade them to perfection ... for example, we would have color corrected in 12 RAW LIN for as long as possible before rendering out the DCP files (or DPX for the film out).

3) We could have then seen the 'stress tests' for each of the cameras and then the 'beauty shots' from each camera of the same set-up .... this would have allowed the audience to really understand the strengths and weaknesses of each camera.

4) It's one thing to show a camera 'stress test' to an audience of skilled DPs who can draw their own conclusions from the way the tests were devised, but to show the tests to a non technical savvy audience of Producers and Directors is somewhat misleading.

The 'Film Look' is still the gold standard for feature films in Hollywood, but the digital 'acquisition+workflow+output' equation is getting much better and is already at a state where great movies are being shot digitally and finished in a optimized RAW workflow.

I'm not sure that the ASC or BSC camera tests conveyed how far RED and RAW has developed and how cost-effective the workflow is now.

Neil

 

[J. P. Sendall]

That being said, Stephen and David... Come on... this test is really ridiculous. They should put a Macbeth chart in the image and grade everything so that the charts match as best as possible. Then we can actually get a sense of the real difference between these formats.

I was at the BSC test and even though I am not a DP I had problems with it. They did use a Macbeth to try and get a "level playing field" was their phrase, but to be honest I would not use what I saw as a tool to decide my acquisition media. I saw 16mm that was really not up to scratch because I've seen a lot better and I saw Red that was way off base in terms of skin tone but I know it does miles better than that. Some have said that the original transfer of the RAW data was the problem before they even got to a minor grade. I don't know enough to judge that but I do believe that as a scientific test it serves no one in what is essentially a creative business. If however they had Macbethed everything and then dropped it all through a grader whose job it was to get the best image possible we could then compare that fairly, not just calibrate to a Macbeth and then shoot regardless. Then I could say that one format performs better in one type of situation than another. But you see even that would not be accurate either if the original data transfer was off and the grade was essentially a rescue rather than a finessing.

It was interesting to see formats up on the screen at the same time that were shot under the exact same conditions but something in the back of my head kept on complaining that I wasn't seeing each format in the best light possible, pun not intended.

 

[J. P. Sendall]

Just to quickly add I have just seen the BSC video on their website and it suggests that producers and directors can use this test to make a more informed choice. I think that is a serious mistake. I would from those tests not decide to shoot 16mm for theatrical release. But then I know "Leaving Las Vegas" was shot on 16mm and looked fantastic. I'm having grave doubts about these tests not because they aren't accurate (and there do seem to be some technical imbalances) but because of the way they are being used.

 

[David Mullen ASC]

You have to keep in mind that some of these formats and cameras only look worse in comparison to each other in certain circumstances... when an entire project is shot using one format or camera, you tend to accept that picture quality and look because you don't have a frame of reference. Hence why 16mm may look worse compared directly to some other formats, but an entire feature shot in 16mm can seem quite acceptable in quality, even attractive.

 

[Stephen Williams]

...but on which stock? The point many have to get their heads around (not talking about people here, just some of the people I come up against day-to-day, post supes etc.) is that a new build on Red is similar to a new stock from Kodak. Shove 1970's filmstock in a modern Arri and you'll get 1970's images (I know, lenses, but they are camera independent). Everyone gets that and we only test the newest stock. Red have designed a digital camera that offers the same kind of (hopefully) constantly open ended on-going development just as if new stocks were being developed but some still make generalisations based on build 13 etc., which is an error. Problem is, this camera is developing so much faster than filmstocks ever have and that seems to make some people suspicious.

Hi,

As all color filmstocks except Vivid 160 are boringly close to each other it really does not matter very much.
Kodak 5254 from the 1970's probably had the best skin tones of any stock IMHO, shame the processing is no longer available.

Stephen

 

[Stephen Williams]

But the audience for this presentation is also producers and directors who I believe could be turned off digital by looking at so many ghastly looking images where as most of the film examples looked good.

Hi Mike,

I am sure those producers will be happy to shoot digital if they can save money. I was under the impression that digital is usually sold as being cheaper, or not requiring lighting (Sony).

Best,

Stephen