HDMI less noisy then footage

[Patrick Tresch]

I understand the compression point but when shooting 6:1 or better, the noise is not amplified by compression. I've never seen compression artefact (on my jobs at least) with REDCODE. But I've seen noise from 1200 asa up. Now I like noise and can live with it... if the projects permits it obviously.

I don't beleive there is a need for another recording device. And I'm a big believer that the one who makes uncompressed RAW for motion picture, make a big mistake. (Hope Aaton will develop some compression ratio in their camera...)

Pat

[Karl H]

I understand the compression point but when shooting 6:1 or better, the noise is not amplified by compression.

Pat

Hi

If I can be of help, I started a thread on this a few months ago. It basically is compression, even noticeable at 4:1 - but you won't notice it if you're nicely exposed. It's really only in low light you'll see it. Simply put, Epic records a signal at around 320ISO (ish) - always. Changing ISO on set does not change the signal recorded, its always 320ISO regardless...So gain boosts, whether 500ISO or 6400ISO are simply 'viewing curves' of a sort applied after compression to a 320ISO image.

Most of the time that's fine, but lets say you happen to be filming low light at 2000ISO. It looks surpassingly clean over HDMI because you're seeing the live signal with no compression. If it was recorded at 2000ISO then it would look great too. But Recode will still record this at 320ISO - in other words, now underexposed. When you play back, the metadata will boost it to 2000ISO but that is simply applying gain boosts to a poor signal during recording (320) . Obviously it will show up the compression, and the more you dig into the shadows (the higher the ISO you view) the more you're pulling out those low light blockies.

However, If you record out of the HDMI live at 2000ISO then you are boosting the signal before recording - so it'll be much cleaner.

Unfortunately it's one of the few drawbacks to Redcode. It records what it sees native regardless of user selected ISO. This is one area where an RGB camera would be superior by boosting the signal first and then recording it - much cleaner in these situations and exactly what's happening if you record HDMI to a samurai. Imagine recording an image on your EX-1 which looked correctly exposed at +12db, but pretty dark at -3db. You would not record at -3db and try to boost it with curves later - but this is how redcode works. They say 800ISO is the sweet spot of Epic, because thats what you can get away with boosting to and still have a clean image in the shadows, beyond that and it's the compression that begins to show up more than the sensor noise IMO.

The next time you shoot anything at high ISO switch the histogram to RAW. That will show you what you're really recording, and in low light situations the whole image can be a small patch on the left side of the histogram. Even at low compression ratios the shadows will always suffer here.

I e-mailed RED about this a few months ago, suggesting that maybe they could look into recording the signal after ISO was applied (but that may take it out of RAW space) or maybe a high ISO mode which puts more focus or bandwidth into the shadows than the highlights. They were actually really nice and said they appreciate feedback and are always looking for ways to improve their products. Fingers crossed they may look into this.

PS I found this ISO thing hard to fathom at first, but it's easy to test.. lets say you have a nice exposed image at 500ISO and within the goalposts on the histogram - before recording now boost it to 6400ISO just for a laugh so it looks bleached out and horrible. Now record. You're image will look F**ked up in the viewfinder true, but it has NOT actually blown (because ISO is irrelevant). You can flip it back to 500ISO on playback and the image will be fine again. ISO is a great tool but always remember your recorded signal can only be viewed by clicking 'RAW'.

[Mark Pugh]

Karl, that's a significant post. It points out something very interesting for anyone shooting low-light. Strange the thread died for a day or two after that..

[Karl H]

I think people just got bored lol.

I was repeating myself too from an old thread. We have been told by Red many times that ISO is only metadata, but I never really understood what they meant until recently. I guess it makes sense, RAW implies no image processing at all, just sensor pickup straight to compression. ISO (or digital gain), is a form of processing so is skipped.

The good thing from all this is that the MX sensor does have the potential to dig into the shadows more than it allows us to right now, you only need to view the live HDMI output to see that. In low light situations you actually DO want the signal boosted before you send it to the compression engine which is why people are seeing better results recording to a Samurai, than to R3D.

IF they could find a way to record the image after digital gain (ISO) then I think we'd all see a massive increase in low light capability. As Jim often says 'it'd be like having a new camera again'.

Perhaps when/if the prores module comes out we can see some tests from that, compared to R3D. My bet would be the prores files would be much cleaner as my understanding is they would be fed RGB after gain applied (assuming the user is shooting in high ISO).

[Jaakko Rinne]

Not sure I buy this as my footage was recorded onto a Samurai with the Apple Pro Res 8 bit codec. I would not think that the Pro Res is any more efficient? As far as scaling is concerned I am comparing the 1080P output of Redcine X encoded with Apple Pro Res to the Samurai recording encoded with Apple Pro Res on the same monitor. Got me baffled why the HDSDI camera output looks cleaner?

It's not really that difficult of a concept to grasp.

1) When you record RAW, you're recording the image signal without any gain applied. So even if you've got a nice full histogram when looking at your footage at ISO1600, you're actually underexposing the sensor by more than two stops and your recorded signal will be largely in the lower end of what the camera can capture.
2) The wavelet compression used by Redcode isn't perfect at low signal levels and will introduce artifacts that make for more pronounced noise.
3) When it comes time to grade, you're lifting the signal from the low end and making the compression/noise visible.

However,
4) When recording from the SDI out, you're bypassing the redcode compression. Your external recorder is getting a "full" signal that you don't need to boost later.
5) Because you're bypassing redcode, your noise will be sharper and less intrusive. There'll still be noise, but it will look better.

But the noise is not due to compression artefact.

Pat

Yes, it is. Or rather, the compression makes the noise look more pronounced.

[Jaakko Rinne]

I understand the compression point but when shooting 6:1 or better, the noise is not amplified by compression. I've never seen compression artefact (on my jobs at least) with REDCODE.

Yes, it is. If you've never seen compression artefacts, you've never really dug into the footage. Try boosting an underexposed area in any shot and looking at the blue channel.

[jimhare]

Yes, it is. If you've never seen compression artefacts, you've never really dug into the footage. Try boosting an underexposed area in any shot and looking at the blue channel.

But is that sensor noise or compression?

[Karl H]

It's compression. The best thing I can say is test. Pipe out hdmi to a 1080p monitor at 2000iso. Compare the live feed to recorded footage of the same scene played back to the same monitor. Even downscaled you'll see the difference. Sensor noise is fine grain and uniform, compression is more random dancing larger blocks with a blue hue.

[Costelloe Michael]

Great thread guys, keep it going. I'm going to ask two obvious questions for clarity.

1. Does HDMI also apply to the HDSDI feed out in the same way or is there some difference in how the signals are put out?

2. I thought Red MX was natively set to 800asa not 320asa, please clarify.

Cheers,

MC

[Karl H]

Hi Costelloe

1. I haven't used HDSDI but Im 99% sure it's identical. Whatever image you see on your LCD can be piped out of the HDMI/HDSDI

2. MX sensor is rated by RED at 800 because that is the level at which you can apply gain and still have a noise free image, but that doesn't mean that the sensor is 800 native. The native sensitivity of the RAW signal is 320. Again it's easy to test... Next time you shoot anything switch your histogram from 800ISO to 'RAW'. You will see the histogram drop back to the left. If you're shooting 320ISO however you'll see very little difference between that setting and the RAW histogram. This said all RGB cameras will have gain applied to them before they go to compression, so rating a camera is subjective. The difference is, on EPIC, the signal recorded is 320. The debate rages on even today about what to rate the MX at, it's about finding a balance between what's acceptable for your conditions. Viewing at 800 gives you more highlight protection because you're actually underexposing the sensor's native pickup. If you choose to view at 320ISO you get less highlight protection because you're not underexposing the sensor.