How RAW is Red RAW? (R3D)

[Ryan E. Walters]

So it is my understanding that what the Red record is RAW (Red's version anyway) and that it is through post that the image is processed. Additionally, each new camera build only really effects how the image is processed in post and it does not effect what the sensor sees - only how it is processed. And this is why Red RAW is so powerful, because I can open footage shot a year ago and process it just like I had shot it today as the sensor is not seeing / recording anything new, just processing it differently.

Is this correct?

I ask because as I do my own testing with the Red M and MX sensor, I am wondering if with each new build I need to reshoot the same tests or can I shoot them once, and then just process them with the new updated software?

With a new sensor, or a different camera I have to shoot a new test as they will physically see things differently. But I am thinking that since these are just new firmware builds, and that since it is RAW, it is only a matter of re-processing the footage rather then running a whole new set of tests for each new build. (I hope this is the case ...)

 

[David Mullen ASC]

All manufacturers' "RAW" recordings have some degree of processing going on even if they aren't converted into RGB color, plus there is REDCODE compression added in the case of Red's RAW R3D files.

But for the most part, yes, you will benefit from taking old RAW files through new conversion software over time. But as your camera gets upgraded over time, there may be features added that will affect what actually gets recorded in RAW (other REDCODE compression level settings are an obvious example.)

With the M-X sensor upgrade there is new image processing software that affects the RAW recording. But in terms of the old sensor, I'm not sure if the various builds have changed how the sensor output itself is processed into R3D files and thus have baked-in certain image improvements that an old file would not have.

 

[Graeme Nattress]

Sometimes a change will change what goes into the files - new sensor, new OLPF/IR in M-X, or compression improvements. But that doesn't stop an old file being improved through new calibration techniques or raw development or colour science.

Graeme

 

[Felix K.]

I have a question: if nothing was compressed at all, how much MB would one picture have? Is it like pixelcount x bitdepth x 3(for the different colourchannel)?
So like

(4000x2000) x 12bit x 3 = 2.880.000.000 bits = 343 MByte?

 

[Graeme Nattress]

Uncompressed, REDOne, is 4096x2304x12bits (bayer pattern, remember so no x3)

Graeme

 

[Julio Quintana]

Uncompressed, REDOne, is 4096x2304x12bits (bayer pattern, remember so no x3)

Graeme

So Uncompressed RAW RED footage would be 13.5 MB/frame and 324 MB/sec? That second number is getting close to the projected specs for EPIC, isn't it?

 

[Alex Lindblom]

Felix for uncompressed RGB you're basically right, but you just missed something in your down conversion.

1 byte = 8 bits
1 KB = 1024 bytes
1 MB = 1024 KB

So one frame would be about 34,3 MB per frame and in 24 frames, 824 MB/s

 

[Graeme Nattress]

EPIC does a lot more fps.

Graeme

 

[Julio Quintana]

EPIC does a lot more fps.

Graeme

Makes sense. But for the sake of my curiosity, which of our calculations is correct?

 

[Graeme Nattress]

Mine.

Graeme

 

[Dan Hudgins]

fps

fps

I've been wondering what happens to the compression ratio at high speed shooting, it would seem that the compression ratio would need to go up?

If EPIC will have enough bandwidth in the recording to record delta-encoded true RAW data at 24fps that would be a great addition, I know you "say" that you do not want to include true RAW recording because there is too much bandwidth for slow motion, but for many films 24fps would do all that is required, so why degrade the images when you don't need to.

The new sensors are 14bit? so 100% lossless delta-encoding will be able to be used with 14bit data to get about 20% to 30% reduction.

4096x2304x1.75=16.5MB/frame * 0.75 = 12.4MB/frame average exposed EI800.

12.4x24=298MB/sec.

298MB/sec is just a little over what the flash module for ACAM seems to do, if you use two flash modules flip/flop bytes you could do maybe 25fps and 30fps as well without the "rolling dishwasher harddrive array" that keeps getting brought up and seems just silly to keep bringing that up since the camera can shoot compressed 100:1 for slow motion since those shots are blured by motion blur anyway, maybe.

The RED ONE produces adequate results, if you look at the uncompressed REDLOG TIF files out of REDCINE you should see that, most problems probably start with how people are processing the files from that point on since you can get double compression losses if you do not post uncompressed all the way to the film recorder. Just go see CHE#1, there were a few spots that looked like compression but those may have been introduced in post, and they say the "black sun" issue has been fixed. For the most part there are few reasons to use film any longer, and since all projection is going JPG2000 compressed it seems, whatever film has above RED is going to get lost along the way maybe in the DI/DCI since film to film printing will be history in a few years.

 

[Julio Quintana]

Mine.

Graeme

That's great. But I tried multiplying the numbers you gave and I just want it confirmed. I've been in several scenarios where professionals are telling me that the RED should have a RAW recorder. I want to show them how impractical that is. Can you give us a straight answer please?

"Without REDCODE, RED Raw would be ___MB/frame"

Thanks,

 

[Dan Hudgins]

Delta or not

Delta or not

"Without REDCODE, RED Raw would be ___MB/frame"

It depends on if you are using delta-encoding or not, since its 100% lossless its the same as RAW data they are 100 percent equal.

Its just one value per pixel for Bayer data, so if you pack the bits like Acam DdII does, for 12bits its 1.5 bytes per pixel, and for 14bit its 1.75bytes per pixel.

With delta-encoding the bandwidth ranges from 50% to 100% the bit packed size with about 70% to 80% being typical, it depends on the exposure and subject matter, if the delta-encoded is going over bit-packed you output bit-packed for those frames.

Another way to think about it is that the true RAW frame is about 1/4 the size of a uncompresed RGB TIF of the same image.

true RAW 4096x2048x1.5=12.6MB/frame (12 bit) [with delta-encoding about 8.82MB/frame]

true RAW 4096x2048x1.75=14.7MB/frame (14 bit) [with delta-encoding about 10.3MB/frame]

TIF (4096x2048x6)+4096 =50.3MB/frame

So 12.6/50.3=0.25.

The 4096 in the TIF is the header size which can vary.

If you record 14bit linear sensor data as 12bit log or gamma corrected you do not loose much that matters so you can reduce the size a little that way, if you go to log 10bit you might start to see a small loss, at 8bit log you would see a loss after grading.

At 8.9MB/frame at 24fps its 214MB/sec or 13GB/minute so 1TB/13GB= 76minutes per TB. or about $1.20 per minute recorded! (vs. $200 or so per minute for 35mm film and lab)

 

[Graeme Nattress]

An uncompressed RED One 12 bit frame 4k 16:9 is 13.5MB, or around 324MB/s. That gives you, should such a CF ever be made fast enough, 25 seconds on a 8GB CF card, rather than that over 4 minutes you get today through REDCODE. REDCODE is just enough compression to make 4k practical, which retaining superb quality through to the screen, as those who attended last week's REDDay can testify.

Graeme

 

[Julio Quintana]

Awesome. Thanks, Graeme.

 

[Steve Sherrick]

If I remember correctly (seems a long time ago now) Jim offered an option to record RAW uncompressed, and he may have even put a price tag on it. I believe no one took him up on his offer. I also remember they did a screening where they A/B'd uncompressed RED RAW vs Redcode and people couldn't pick out which was which for the most part. So, seems Redcode won out. I suspect the new, higher data rate codecs will look better in certain situations but perhaps not as dramatically as some might guess. I'm all in favor of less compression, especially when one can show me a noticeable difference in quality. Even then, it becomes a question of economics. I probably can't afford to shoot in a Red Raw uncompressed codec. I just don't have the storage and backup infrastructure to handle it yet. Redcode is efficient and the results are great. A lot of things I might see, nitpicky stuff, clients will often never see.

 

[Julio Quintana]

I have no use for uncompressed RAW. As a matter of fact, I have been in several ENG type situations where I would have killed for a 1080p ProRes recording format.

 

[Ryan E. Walters]

But for the most part, yes, you will benefit from taking old RAW files through new conversion software over time. But as your camera gets upgraded over time, there may be features added that will affect what actually gets recorded in RAW (other REDCODE compression level settings are an obvious example.)

Sometimes a change will change what goes into the files - new sensor, new OLPF/IR in M-X, or compression improvements. But that doesn't stop an old file being improved through new calibration techniques or raw development or colour science.

Graeme

Thanks David and Graeme for your replies. So if I am following you correctly, the short answer is that yes I do need to redo various testing that I've done in the past on new builds, even when using the same sensor, as the behind the scenes magic does qualitatively change how the sensor responds, and it is not just all a post processing issue.

 

[Graeme Nattress]

On some builds yes, that is the case. On M v M-X absolutely the case. For other changes such a the colour science, no, you don't need to re-shoot, just re-process.

Graeme

 

[Ryan E. Walters]

On some builds yes, that is the case. On M v M-X absolutely the case. For other changes such a the colour science, no, you don't need to re-shoot, just re-process.

Graeme

Great- good to know. Moving forward with the MX sensor and the new builds that will be coming out for that, is there a way to tell when I should re-shoot, verses re-process?