It ain't the camera: It's YOU !

[David Battistella]

That is a great point Matt. I notice it happens in that area above black, like those trees, which represents a grey value so that noise is not happening in the blacks but in that kind of misty part that would be a grey value sitting between 8-20 ire on the waveform.

If this is the case, the meters are reporting correctly for the extremes, highs and lows, but that area just above black can get noisy particularly in high detail scenes. Maybe time to record some greyscale charts again to see what is going on.

 

[Matt Ryan]

That is a great point Matt. I notice it happens in that area above black, like those trees, which represents a grey value so that noise is not happening in the blacks but in that kind of misty part that would be a grey value sitting between 8-20 ire on the waveform.

If this is the case, the meters are reporting correctly for the extremes, highs and lows, but that area just above black can get noisy particularly in high detail scenes. Maybe time to record some greyscale charts again to see what is going on.

Thanks David, this has been boggling my mind for a while. Because theoretically it should be clean and free of noise, yet I often see noise even when it isn't underexposed.

 

[Scott Crawley]

Heres the thing though, if goalposts always represent the RAW sensor data, than if neither goalposts are lighting up, why is there noise in the frame? If the RAW sensor has not been clipped or crushed, you should have a clean signal to begin with. So why is Will's shot for example noisy in the blacks? Of course after processing you can cause one or the other by lifting the low end (etc.), but theoretically it should be crisp and clean on both ends.

I think this is where the confusion lies. Because obviously the goalposts are incorrectly calibrated if you can have noise in the low end, but absolutely no signal in the goalposts stating the RAW data has noise or underexposure... Regardless of if you ETTR or not, the signals should reflect accurate data. I have ran into this issue before shooting at 320 ISO and neither goal post is lit up, yet when I bring it into RCX, theres dancing noise (although not bad) in the dark areas. This shouldn't be happening at all. If the RAW sensor isn't crushed or lacking info, there should be no noise... (And yes I light and expose properly...)

Obviously there are things you can do like use grad ND's in Will's situation, but honestly there shouldn't be noise there.

With those things in mind you look to compression ratios, as you already have, and black shading. Given the fine detail in the shot I also lean toward compression, particularly if the if the lens may not be resolving sharply in those shadows for whatever reason; missed focus or DOF etc..

What other explanation might there be?

 

[Matt Ryan]

With those things in mind you look to compression ratios, as you already have, and black shading. Given the fine detail in the shot I also lean toward compression.

What other explanation might there be?

That's why I said earlier that REDCODE needs to be tweaked a bit to handle high detail scenes better. It's a miraculous thing but it has it quirks. It's just frustrating when your shooting at 5:1 and seeing compression break down in scenes like this. Even shot on a busy street during the day with trees and people and buildings breaks down every once in a while. It's unfortunate that you expose properly and still have issues. I shot a well exposed green screen shot at 100fps and could see the blocky compression in the actors faces even though they were properly exposed at 320ISO & 5600K. So theoreticall that isn't user error. It's situations where compression breaks down that the operator can't really be blamed.

 

[Scott Crawley]

That is a kind thought, but we are ultimately responsible. We should be alert to the fact that scenes with more abundant fine detail require lower compression rates. As a rule of thumb I subscribe to no more than 3:1 for most wide vistas. It has served me well so far.

Any shot in which I am striving to capture detail rich shots... those times that I am particularly thankful to be shooting 4k... when I know I want to utilize every bit of resolution the system can muster I say to myself "Lower your compression rate." At least I try to remember. (Again I'm talking fine details, not big close ups of faces and the like.)

It is relatively easy for me to remember because I recall screening an HD short I shot at a festival. I was excited to see how our HDV system and 35mm adaptor had worked out on the big screen. I was disgusted to see macro blocks. Then other films showed the same and the queasy feeling went away, but I remember it well.

That's why I said earlier that REDCODE needs to be tweaked a bit to handle high detail scenes better. It's a miraculous thing but it has it quirks. It's just frustrating when your shooting at 5:1 and seeing compression break down in scenes like this. Even shot on a busy street during the day with trees and people and buildings breaks down every once in a while. It's unfortunate that you expose properly and still have issues. I shot a well exposed green screen shot at 100fps and could see the blocky compression in the actors faces even though they were properly exposed at 320ISO & 5600K. So theoreticall that isn't user error. It's situations where compression breaks down that the operator can't really be blamed.

 

[Matt Ryan]

At high frame rates you don't have that option. So that is not user error. Plus not everyone who shoots a film in the Forrest have the disk space for 3:1. Redcode should hold better in these situations. As much as the Epic is superior, other cameras in its competitive range don't break down like that, although I know they aren't processing this high of resolution and frame rates. All I'm saying is it needs work and ultimately is not the fault of the operator when the codec breaks down. Sure you can shoot 3:1 in some instances, but not all.

 

[Mark Pugh]

I hear you Matt, and it's a real worry with 6k coming up.
I find it easy to imagine that high frame rate 6k is going to be difficult in many situations, with the current technology in the Epic, and the promised faster SSDs and better compression haven't really materialized yet. If the new ASICs don't work some wonders, there may be interesting debating soon regards resolution vs processing.

 

[Scott Crawley]

Other systems are holding up better? I see what you mean then, though my own experience with other systems and higher frame rates is somewhat limited.

I appreciate your distinction as to whether or not it is user error in the instance that no setting could be changed to improve the situation. On the other hand, an individual who chooses to shoot Epic at high resolution without the required storage is not without blame, and HFR shooting remains a specialized skill.

I don't know if I am so inclined to deride Redcode over this. As you say, the resolution is high and the frame rates are high. I too think Redcode compression is pretty miraculous! If I were looking to improve my camera system's HFR performance, I would likely be looking at the factor limiting my ability to choose lower compression ratios at higher frame rates. This isn't as simple as the size of a mag, is it?

At high frame rates you don't have that option. So that is not user error. Plus not everyone who shoots a film in the Forrest have the disk space for 3:1. Redcode should hold better in these situations. As much as the Epic is superior, other cameras in its competitive range don't break down like that, although I know they aren't processing this high of resolution and frame rates. All I'm saying is it needs work and ultimately is not the fault of the operator when the codec breaks down. Sure you can shoot 3:1 in some instances, but not all.

 

[Scott Crawley]

In this highly particular circumstance: large vista high detail high frame rate shooting, maybe 65mm+ is still the only viable option for now.

I'm betting that Jarred will be trotting out the next incarnation of Redcode along with Dragon that will fit the bill. What do you think?

 

[Matt Ryan]

Other systems are holding up better? I see what you mean then, though my own experience with other systems and higher frame rates is somewhat limited.

I appreciate your distinction as to whether or not it is user error in the instance that no setting could be changed to improve the situation. On the other hand, an individual who chooses to shoot Epic at high resolution without the required storage is not without blame, and HFR shooting remains a specialized skill.

I don't know if I am so inclined to deride Redcode over this. As you say, the resolution is high and the frame rates are high. I think the data rates over it's compression range are pretty miraculous! If I were looking to improve my camera system's HFR performance, I would likely be looking at the factor limiting my ability to choose lower compression ratios at higher frame rates. This isn't as simple as the size of a mag, is it?

I enjoy healthy discussions like this. It makes us all better and helps improve the products. I shot some Alexa footage at 120fps and it was clean, no macro blocking or compression noise. Same with the FS700 that I just used the other day, although I didn't zoom in 200% on the FS700 to look. I just think it's a little disappointing to shoot everything perfectly at high speed or detailed scenes at as low a compression ratio as possible and still have blocking of some sort. The weird thing is sometimes it's there, sometimes it's not. Even in the same location, same settings, same everything. That also blows my mind. I use RED cameras whenever I can because of all the benefits and superior resolution, but it's a little embarrassing when you talk the camera up then play back the footage on a 2K display and see compression blocking in perfectly exposed footage.

I really hope that with Dragon they have improved the compression code/processing to get things a little cleaner or at least make the compression blocking look more like fine grain or noise than the larger blocking. This would make it look a lot better. With 6K it will be harder on the compression, so hopefully things don't get worse. I love shooting high speed, and at 5:1-7:1 compression for normal situations, just wish REDCODE held up in every situation. The compression blocking is ugly...

 

[Matt Ryan]

I hear you Matt, and it's a real worry with 6k coming up.
I find it easy to imagine that high frame rate 6k is going to be difficult in many situations, with the current technology in the Epic, and the promised faster SSDs and better compression haven't really materialized yet. If the new ASICs don't work some wonders, there may be interesting debating soon regards resolution vs processing.

Your absolutely right. I'd rather have 5K with Dragon (16 stops DR), and better compression ratios at high speed and all around. The thing is, if you shoot 6K and have compression blocking mucking up the scene, does it really matter whether it's 6K or not? I'm betting they already thought of this and will improve things. Otherwise, things will be very interesting.

Considering the new .RED codec is said to provide great detail even in gradients and high detail, I really hope that means they have improved Redcode for Dragon.

 

[Scott Crawley]

I enjoy healthy discussions like this. It makes us all better and helps improve the products. I shot some Alexa footage at 120fps and it was clean, no macro blocking or compression noise. Same with the FS700 that I just used the other day, although I didn't zoom in 200% on the FS700 to look. I just think it's a little disappointing to shoot everything perfectly at high speed or detailed scenes at as low a compression ratio as possible and still have blocking of some sort. The weird thing is sometimes it's there, sometimes it's not. Even in the same location, same settings, same everything. That also blows my mind. I use RED cameras whenever I can because of all the benefits and superior resolution, but it's a little embarrassing when you talk the camera up then play back the footage on a 2K display and see compression blocking in perfectly exposed footage.

I really hope that with Dragon they have improved the compression code to get things a little cleaner or at least make the compression blocking look more like fine grain or noise than the larger blocking. This would make it look a lot better. With 6K it will be harder on the compression, so hopefully things don't get worse. I love shooting high speed, and at 5:1-7:1 compression for normal situations, just wish REDCODE held up in every situation. The compression blocking is ugly...

Agreed, and your experience is indeed perplexing. I am particularly interested in this part "play back the footage on a 2K display and see compression blocking in perfectly exposed footage." This seems like a simple question but have you examined that same footage on a proper 4k screen? What method are you using to down sample for 2k display? Is there any possibility that is at fault?

 

[Matt Ryan]

Agreed, and your experience is indeed perplexing. I am particularly interested in this part "play back the footage on a 2K display and see compression blocking in perfectly exposed footage." This seems like a simple question but have you examined that same footage on a proper 4k screen? What method are you using to down sample for 2k display? Is there any possibility that is at fault?

I haven't had a chance to see it in 4k unfortunately. Would love to. Playing back in RCX Pro with a RedRocket (Full res), on a calibrated Apple Cinema Display 30". I shot a video for MUSE on a green screen stage and the drummer was doing a jump towards camera from a trampoline, and there was compression blocking dancing on his face and body (not sensor noise or grain but that blurry macro blocking). Shot it at 100fps, and at 320 ISO, histogram perfectly centered with no clipping or crushing. Yet still had the issue. Looked great on the monitor in live view, but as soon as it's compressed it got blocky. It's not terrible, but it is definitely noticeable and wasn't visible in live view. So I'm sure REDCODE was the culprit. At 1080p it isn't that noticeable, but viewing in higher res it definitely is.

 

[Scott Crawley]

I would love to hear from one of our systems people on this. Could there be a bottleneck happening in Matt's system that would cause what he is seeing... GPU, thru-put? Kilgroe, anyone? It seems feasible.

I haven't had a chance to see it in 4k unfortunately. Would love to. Playing back in RCX Pro with a RedRocket (Full res), on a calibrated Apple Cinema Display 30". I shot a video for MUSE on a green screen stage and the drummer was doing a jump towards camera from a trampoline, and there was compression blocking dancing on his face and body (not sensor noise or grain but that blurry macro blocking). Shot it at 100fps, and at 320 ISO, histogram perfectly centered with no clipping or crushing. Yet still had the issue. Looked great on the monitor in live view, but as soon as it's compressed it got blocky. It's not terrible, but it is definitely noticeable and wasn't visible in live view. So I'm sure REDCODE was the culprit. At 1080p it isn't that noticeable, but viewing in higher res it definitely is.

 

[Craig Parkes]

I'm going to add to this thread with a few thoughts.

Sometimes the biggest problem an emerging film maker has is expecting results they simply can not get with the tools they have, and then blaming the tools they have rather than the lack of other tools they need. Or trying to get the MOST out of the gear they have (in this case shooting high frame rates with little light control on the highest possible f stop of the lens on a heavy sensor crop etc) and thinking the tool is to blame when there are all those compromises already going into the image making process.

Maybe the lesson here is just for Will to ask "Could a NDs/Grads helped save the original shot?" Rather than 'When will Epic/Any other camera be able to capture this scene cleanly, without any noise or compression?"

Because honestly, creatively I have questions about this shot. Like, should the shot have even been attempted at 300FPS?

Did you really NEED to be shooting 2K at 300FPS, or could it have been shot at 150FPS - creatively what's the need of a 300FPS image here (can't tell from the still) - would you have been better at 150FPS and slowing down in post to get the desired effect given the conditions?

(I find high frame rate is a HUGE issue in this regard, because people aren't used to shooting it and they think the images should just come out like regular images, when they should know going to high frame rates starves the camera of light. Shooting cropped sensor 2K to get to 300 FPS is like removing half the weight from a race car to get it to go to 300mph - even when it works, you know you are compromising every other aspect of safety you rely on to get to that top speed - which only makes sense when you that speed is necessary and the compromises aren't lethal.

Don't know what exactly is happening in this situation, other than the obvious issue of compression. Even though compression artifacts are arguably a situation where the tool itself isn't perfect, it's still not a FAILING of the Epic - no camera advertises itself as perfect for all uses, in all situations, always.

Just like if you had tried to do the same thing with a non high speed film stock the shot would have been ruined, you wouldn't blame the film stock for those results, you'd blame the production for not getting the right stock, AND the DOP for then choosing to 'get away with it' rather than changing tact and making the right compromise based on conditions.

Knowing when you are pushing the envelope is key - So Will, creatively, in this shot, why were you shooting at the extremes of the cameras (and your lenses) capabilities? What in that shot were you trying to capture that required 300FPS motion, and what motivated you to go out in the field without an ND kit for the camera, etc, and what's so vital in the darkness off that treelike that you need to be able to see it, when the shot isn't exposed for the trees?

Any argument against 'it's not the camera, it's you' doesn't work when you are the one planning and prepping the shoot, and figuring out what you need to take out with you into the field to get the results you want. That's STILL YOU, unless what you NEED to capture is simply impossible with current technology - in which case, you get creative and fake it (which is kinda the point of film making...)

My point is this: Getting the results YOU NEED with the tools you have is the sign of an experienced DOP, getting a result with unusable issues with the tools you have is the sign of an inexperienced DOP (or a lack of familiarity with equipment). Aiming for a result you WANT, but failing to get a usable result is a failed experiment.

For the record - lack of experience isn't a bad thing (unless you are selling yourself on the basis of being experienced) - We are all always learning, so it's not derogatory to point out the fact that you are inexperienced with these tools - hell, almost everyone is. What I feel the original poster is really talking about when they say learn to shoot, is learn about ALL the tools you need for making great images - not just your camera.

Expecting your tools to be perfect is a bad thing, because by and large they never are - every lens, every camera, every light meter, every hammer or screw driver or pencil has plusses and minuses - sometimes the answer to 'It doesn't look good when I try and get this shot!" is to try and get a different, BETTER shot with the tools you have.

Rather you should be creatively selective about the shots you spend your time taking with the tools you have, and when you fail to get what you want with the tools you have look at it as a failed experiment on your behalf, rather than a failing of the tool.

If this was film - every failed experiment would cost you real hard cash, so take solace in that even if the EPIC isn't quite the perfect camera - it doesn't cost you anything other than time to figure out how to get the best shots out of it based on the tools you have - and that in most cases the compromises you are forced to make based on camera/sensor/negative are more or less the same and sometimes LESS than those forced on every other DOP in the history of the world - without all the costs attached to shooting/learning that film provided.

 

[Matt Ryan]

If no one should be shooting 300fps because it is borderline unusable, it shouldn't be an option in camera then. It's there for a reason, to be used. The problem even when correctly exposed is the compression. That is not user error.

In Will's case he should have used ND, and he shouldn't have been shooting at t22. But the point is if all the sensor data is there, even though he didn't take the proper steps to get perfect images, it should still be clean and noise free. Whether the trees are important or not there shouldn't be blurry blocking going on. While he could have done other things to get a proper image, that blocking would most likely still be there. No one is saying he shot it properly, but the fact is we shouldn't be seeing compression blocking. That is a limitation of the camera itself, independent of the user in this case.

I graduated from film school, trained of film and digital, and had a well rounded education on all the tools and proper shooting procedures. None of that can prepare you for compression artifacts regardless of how well you light, expose, filter, etc. Even if you nailed everything perfectly, you can still run into compression issues. So really the compression is what we should be focusing on.

No one is saying the camera sucks or any other camera is superior, just that the compression sometimes gets in the way regardless of how much care you take. I know for a fact this has happened to a lot of people. If you check the links I posted earlier there have been quite a few tests done regarding REDCODE compression. Interesting stuff. I know in my example everything was exposed correctly at 320 ISO balanced with daylight fixtures, and at 100fps (for a creative reason), and still had the compression blocking. Your still pulling off a good shot, it's just not as clean as one would want. As a user I did everything I could to get a clean pristine image, yet got compression blocking. Yes, I knew of the compression limitations but needed the shot for the video so nothing you can do.

I do believe that 97% of camera issues these days are user error. You can make beautiful images with pretty much any camera. But each has a quirk, and in this case it's compression.

 

[Marc Wielage]

Sometimes the biggest problem an emerging film maker has is expecting results they simply can not get with the tools they have, and then blaming the tools they have rather than the lack of other tools they need.

I really like your message, Craig -- it should be a permanent sticky on the forum for people to read whenever they have questions along these lines. You speak very wisely.

 

[Scott Crawley]

If no one should be shooting 300fps because it is borderline unusable, it shouldn't be an option in camera then. It's there for a reason, to be used. The problem even when correctly exposed is the compression. That is not user error.

I had considered that response, but it depends on several factors and some of those are subjective. This isn't an issue for all shots at 300 fps, only specific types of scenes. Some folks are getting performance from it @ 300. Why hobble the camera? Why design for the lowest common denominator? I'm overstating of course, but to make a point.

 

[Matt Ryan]

I had considered that response, but it depends on several factors and some of those are subjective. This isn't an issue for all shots at 300 fps, only specific types of scenes. Some folks are getting performance from it @ 300. Why hobble the camera? Why design for the lowest common denominator? I'm overstating of course, but to make a point.

Yeah I understand, I'm not saying hobble the camera. I was just speaking in response to Craig saying why would you need 300fps and that it's basically unusable. That's why I said if it's unusable why is it there?

I've gotten good results at 300, and other times terrible results. I just think compression again plays a huge role depending on the scene or the subject being shot. I'll still shoot RED, even knowing everything about it, I just think we need to push for improvements. People already have a negative connotation of RED and look for every little quirk to pick on it. Rather than call them idiots or biased or uneducated, I think we should work on these issues. Then what could anyone complain about? We can't sit here and say it's not an issue, because eventually other cameras will catch up, and the question is will they do it better? In order to stay ahead we have to keep improving things. We as a community need to call out it's quirks so they can be fixed. It benefits everyone, seriously.

 

[David Battistella]

Just want to say I am really enjoying this discussion and what excellent points are being made in a rational way.

Matt, you are correct, compression is a tricky thing to get right. It's a chicken and egg thing for red and us. We want higher frame rates, the camera can accomplish them in most cases with higher compression (more compression) and new issues can emerge.

Its not like film in this regard because (and I know you already know this but I am writing this for the benefit of others) REDcode is not a constant, it is a variable.

A fresh 250iso daylight stock is a just that and it's constant until it becomes stale dated. So if you put it through a film camera at 1, 24, or 150fps, it's going to react the same way to light.

Of course the red sensor is going to act the same way as the film to light, but in order to store it in the raw container, or capture it at a high speed, that data rate needs to be managed.

Im glad we have the choice and that we can exchange strategies on how to avoid issues. If RED had said, it's always 5:1 and limited all the other modes or creative choices, I think we'd be saying different things (like shit, just let me shoot 300fps at 12:1 and ill deal with it)

i appreciate all side on this and what you have contributed to the conversation.

The others are making a great contribution as well.

david