ProRes and Weapon

[Björn Benckert]

I have a really difficult time decoding what the issue was until I did some digging and found that REDCODE encodes linear 16-bit data to a 12-bit value. The documents I've found validating the 12-bit limit of REDCODE all predate the DRAGON sensor. In any case, I think Björn's point is that dropping from 16 bits to 12 bits and then compressing and then downscaling is likely to be worse (by a lot?) than staying at 16 bits through the chain.

I've done my own pixel-peeping at a slightly different question, which is whether it is better to shoot a smaller format (UHD) with lower compression (7:1) or a larger format (6K) with higher compression (16:1) and then down-sample, and the answer is "it depends". But what I don't understand is if we keep the bit depth constant and we keep the overall data rates consistent, are we still talking about whether its best to compress first and then downscale or downscale and then compress? If so, I don't really know the answer. I can believe that scaling 16 bit data to fit a 12 bit container is going to be lossy, especially in the shadows.

No it's not so much about the bits 16 or 12. Even though 16 bit quicktime straight from the chip is something new, that we have not had before, and that is very close to what red calls "RAW" so that makes the difference between "RAW" and quicktime much smaller, we only had 10bit quicktime before. XQ is somthing else.

But the benefits of in camera downscalings is also the following:

You have picture coming from the chip, let say 8k resolution. Then you have the data limits of the rest of the chain, the cards and what not, that demands that we shoot compressed r3d's at most or all framerates.

An 8k signal is a lot of data to compress, and if the signal is not downsampled it will have quite a bit of noise in lowlights etc that is hard to compress.

Now if weapon takes this 8k stream, downsample it to 4k and then compress it. The signal noise is first interpolated to 1/4 of the original noise per pixel level. Also the image is then only 1/4 of the size of the original, then the bit rate per pixel can be four times as high compared to what the original r3d 8k could possibly hold.

So the UHD proress file will be 1/4 of the noise and 1/4 of the size at the time it gets compressed. A cleaner image with more datarate per pixel. And those 1/4 noise and 1/4 size is kind of exponential or multiples with each other so the difference is far more than four times better... Don't know the exact numbers but my guess would be atleast shitloads better blacks. Like really high ISO good blacks.

So in other words an image far more ease to compress and as the stream is saved with the same bit rate limitations of the card as for the r3d, well then this means far cleaner blacks, and as that UHD quicktime is 16 bit there is no real loss in colors etc if done right.

It's the same down scaling trick that was/is done when shooting HD proress at 10 bit with an Alexa. But Red is about to do the same downscaling trick but at a higher level 8k to 4k and with a higher bitrate, 16 instead of 10.

 

[Nick Morrison]

No it's not so much about the bits 16 or 12. Even though 16 bit quicktime straight from the chip is something new, that we have not had before, and that is very close to what red calls "RAW" so that makes the difference between "RAW" and quicktime much smaller, we only had 10bit quicktime before. XQ is somthing else.

But the benefits of in camera downscalings is also the following:

You have picture coming from the chip, let say 8k resolution. Then you have the data limits of the rest of the chain, the cards and what not, that demands that we shoot compressed r3d's at most or all framerates.

An 8k signal is a lot of data to compress, and if the signal is not downsampled it will have quite a bit of noise in lowlights etc that is hard to compress.

Now if weapon takes this 8k stream, downsample it to 4k and then compress it. The signal noise is first interpolated to 1/4 of the original noise per pixel level. Also the image is then only 1/4 of the size of the original, then the bit rate per pixel can be four times as high compared to what the original r3d 8k could possibly hold.

So the UHD proress file will be 1/4 of the noise and 1/4 of the size at the time it gets compressed. A cleaner image with more datarate per pixel. And those 1/4 noise and 1/4 size is kind of exponential or multiples with each other so the difference is far more than four times better... Don't know the exact numbers but my guess would be atleast shitloads better blacks. Like really high ISO good blacks.

So in other words an image far more ease to compress and as the stream is saved with the same bit rate limitations of the card as for the r3d, well then this means far cleaner blacks, and as that UHD quicktime is 16 bit there is no real loss in colors etc if done right.

It's the same down scaling trick that was/is done when shooting HD proress at 10 bit with an Alexa. But Red is about to do the same downscaling trick but at a higher level 8k to 4k and with a higher bitrate, 16 instead of 10.

Could we get those blacks in native R3D? In post. Using a RR?

 

[Christopher*Seguine]

You all know Prores XQ can hold 16 bit....

No, some of us might not, because its not true.

ProRes 444 XQ is just 12bit, just like ProRes 444,it just has higher target bitrate.
ProRes 444 and ProRes 444 XQ support a mathematically lossless Alpha Channel at 16bit, maybe you are confusing that with image data that is 12bit.

So the speculations here is what is the better option between:
1. r3d raw compressed capture (at lets say 8k or 6k) then downscale to UHD or even 2k.
or.
2. Dragon sensor material (probably going in a different path, not trough r3d) down sampled before compression and then saved into a 16 bit proress file.

1. You are neglecting the entire advantage of redcode, its compressing raw data, not RGB data.

Compressing to Prores is inflating the source to RGB and compressing 3x the amount of data

For example, Dragon maxs out 6k/4:1 at 175MB/sec

6144 x 3160 x (12bit/8) = 29.1 MB/frame
/4x compression = 7.3MB/frame *24 (+.5mb for audio) = 175MB/sec

if you were to downsample that to 4k prores ,
4k ProRes 444 XQ 4096 x 2160 x (36bit/8) = 39.8 MB/frame
/3x compression = 8.8MB/Frame * 24 = 212 MB/sec

ProRes XQ would be a much higher data rate for much less information - and that is ignoring any compromises made by realtime debayer and scaling.
The output of the redrocket/x is significantly worse than cpu/gpu processing - trying to do that in-camera is probably going to be even worse.

2. There is no point in recording 16bit data, Dragon sensor is maybe 14bit, probably only 12bit, who knows since they do not publish specifications like Arri and Sony. The least significant bit is garbage, so it makes sense that Redcode is only 12bit. Arri's trick is to map the 14bit raw data to 12bit log for a cleaner recording, Red has no such trick, they have stated redcode is linear.

 

[David Battistella]

Let's talk a minute about what you give up with 4K prores.

1. Ability to change ISO AFTER capture
2. Ability to change or set WB AFTER capture
3. 6K reframing
4. Slow motion?
5. Certain frame rates and data rates.
6. HDR X (pretty sure is only an R3D stream)
7. Improvement over time (new color science benefits even my old footage)
8. File size
9. Flexibility to change compression based on situation. (I capture interviews almost always at 18:1)

The more I think of Prores in camera, the more I understand the initial reasons I did not care for it much.

 

[Antony Newman]

With Prores XQ - Dynamic range may now be great enough to encapsulate Usable Weapon DR.

RED colour granualarity for R, G & B channels is 16 x Prores.

For Rec709 deliverables - 12bit RGB capturing in Prores XQ should be good enough.

For REC2020 or HDR deliverables, 12 bit probably insufficient. I thought that Doldy discussed 14bits per channel only being sufficient for partial Rec2020 coverage with HDR.

Summary : I believe that a Dragon sensor in Weapon body captures more usable colour information (ie a fatter colour negative) than can be expressed in Prores XQ.

AJ

 

[Mathieu Ghekiere]

3. 6K reframing

Not talking about the ability of Weapon to write a 6K Prores, and the file size or the other things you mentioned (in which you are completely right of course), Prores is resolution independent, so you should be able to make a 6K Prores too.

 

[David Battistella]

Not talking about the ability of Weapon to write a 6K Prores, and the file size or the other things you mentioned (in which you are completely right of course), Prores is resolution independent, so you should be able to make a 6K Prores too.

This was my point in starting the thread. I don't think the data rates will support the dual recording needed to do 6k Prores and R3D, but maybe if the camera has a QT only option then it would be best to pass the frame size as selected in the camera right through to the Prores file, thus eliminating any need for onboard scaling and processing. But this would also mean that a Weapon could record without recording an R3D which would be the end of an era as the first RED camera that could theoretically NOT record R3D.

Why Limit to 2K when you could do 6K. I'm told that there is no spec for 6K prores, but who know, things change.

 

[Antony Newman]

Apple likely license the Prores standard to keep the maximum resolution to be in-line with the highest resolution screen they offer in OS-X.

If Red and Apple are working together ... This bodes well for the future of Prores.

Perhaps a ProRes 8K 'XQ' format will come out .... ?

AJ

CORRECTION : "All Apple ProRes codecs support any frame size(including SD, HD, 2K, 4K, and 5K) at full resolution. The data rates vary based on codectype, image content, frame size, and frame rate."

 

[Mathieu Ghekiere]

Apple likely license the Prores standard to keep the maximum resolution to be in-line with the highest resolution screen they offer in OS-X.

If Red and Apple are working together ... This bodes well for the future of Prores.

Perhaps a ProRes 8K 'XQ' format will come out .... ?

AJ

Sorry Antony, I think you are wrong
I just, for testing purposes, wrote an 9760x5000 Prores out of FCPX on a Macbook Air. No problem. There is no screen that Apple offers that is that resolution.
Unless I am understanding you wrong...
5K Prores was already in their spec sheet well before they offered a 5K Retina. Probably because a lot of people were shooting 5K on an Epic, and they wanted to give some idea about datarates.

If you want to write an 8K Prores XQ file, you can do it now already. I would think that if you license the codec, and you have the horsepower in your camera and the media, you should be able to write a 8K XQ file out of the box? (I don't want to know the size of those files though ) Again, am I misunderstanding something?

 

[Björn Benckert]

No, some of us might not, because its not true.

ProRes 444 XQ is just 12bit, just like ProRes 444,it just has higher target bitrate.
ProRes 444 and ProRes 444 XQ support a mathematically lossless Alpha Channel at 16bit, maybe you are confusing that with image data that is 12bit.



1. You are neglecting the entire advantage of redcode, its compressing raw data, not RGB data.

Compressing to Prores is inflating the source to RGB and compressing 3x the amount of data

For example, Dragon maxs out 6k/4:1 at 175MB/sec

6144 x 3160 x (12bit/8) = 29.1 MB/frame
/4x compression = 7.3MB/frame *24 (+.5mb for audio) = 175MB/sec

if you were to downsample that to 4k prores ,
4k ProRes 444 XQ 4096 x 2160 x (36bit/8) = 39.8 MB/frame
/3x compression = 8.8MB/Frame * 24 = 212 MB/sec

ProRes XQ would be a much higher data rate for much less information - and that is ignoring any compromises made by realtime debayer and scaling.
The output of the redrocket/x is significantly worse than cpu/gpu processing - trying to do that in-camera is probably going to be even worse.

2. There is no point in recording 16bit data, Dragon sensor is maybe 14bit, probably only 12bit, who knows since they do not publish specifications like Arri and Sony. The least significant bit is garbage, so it makes sense that Redcode is only 12bit. Arri's trick is to map the 14bit raw data to 12bit log for a cleaner recording, Red has no such trick, they have stated redcode is linear.

Hmm, Been told XQ can hold 16 bit. But possibly not so. And your math above is probably right, and possibly the RED is only 12 bit theory as well. And if so what I'm been saying is not holding water for UHD. But seeing what a clean record it's possible to make trough the HD SDI 8bit out of current dragons compared to r3d capture and then in computer downsample to HD 8bit. I still have quite high hopes that internal proress recordings is not only intended for proxy use.


And yes I'm not saying r3d or "RAW" is useless and no good but UHD or 2k Proress files will at times handle noise better if they are made right from 6k maybe not at UHD but at 2k and coming from 8k... well then UHD proress with post debayer and downsample compression would win when there is a lot of noise I think. White balance, ISO, etc aside.So yes offcourse if you capture a proress file at 50ISO and then in post decide you want it in 12500ISO, well then the baked in digital minus gain will hit you but if the strive is to get the least noise from full sensor 8k to a 2k finnish for an under exposed night shoot, then recording proress at 12500ISO at 2k most likley will beat the shit out of recording r3d at 8k and then later downsample.

 

[Björn Benckert]

Sorry Antony, I think you are wrong
I just, for testing purposes, wrote an 9760x5000 Prores out of FCPX on a Macbook Air. No problem. There is no screen that Apple offers that is that resolution.
Unless I am understanding you wrong...
5K Prores was already in their spec sheet well before they offered a 5K Retina. Probably because a lot of people were shooting 5K on an Epic, and they wanted to give some idea about datarates.

If you want to write an 8K Prores XQ file, you can do it now already. I would think that if you license the codec, and you have the horsepower in your camera and the media, you should be able to write a 8K XQ file out of the box? (I don't want to know the size of those files though ) Again, am I misunderstanding something?

Yes, also autodesk Flame gives the option to use XQ even for internal use on the stone file system and I dont remember what the max resolution is but it's in the 8 or 10k range. And it does so in 16bit float, but possibly thats just in the nodes an the actuall files created is only 12bit as was stated here.

 

[Nick Morrison]

Bjorn how clean do think Weapon Prores will be? A7s clean?

 

[Elsie N]

Wasn't Jarred all giddy from driving around in his vehicle with a Weapon 8k shooting street scenes at night?
Kinda led me to believe noise (in 8k, anyway) is a thing of the past, no matter the output.

 

[Nook Kim]

Very good thread indeed. It's funny how many people think all shooters and post people know in's and out's of R3D, let alone its benefits. There are so many that still don't know or care about R3D, and when many of these people are faced with R3D's from productions, some even talk production into going with other systems than R3D.

I've been working with R3D's since the beginning, but it always baffles me why Red haven't implanted Prores (scaled) and win over ridiculous number of customers from Alexa.

Prores 2K has one more disadvantage in my dictionary even as proxy.. it's not high quality enough for judging sharpness so when you conform to R3D you may end up with lots of soft images that you otherwise wouldn't cut in if you had 4K or above Prores. If Red is going to do Prores, 4K is the least necessity in my opinion.

Also, I'd really like to see 4K HDMI out of Weapon body itself too. Thanks.

 

[Björn Benckert]

Bjorn how clean do think Weapon Prores will be? A7s clean?

One can only hope, speculate and guess. A7s is a different sensor and I can be totally wrong on this, as I have not tested it or even know how the signal chain goes for wepaon. But a 4 to 1 down sampling, 8k to 4k on a Cmos sensor without compression before debayer and down sampling is atleast to me very interesting.

A test for reference would be to shoot dragon at 6k and frame burst or slow frame rate so compression can be down to a minimum and then scale down the image to 6/8k the "glossiness" of that image would equal 8k capture to 2k process I guess.

 

[Antony Newman]

Thanks Mathieu.

I skimmed that part of the white paper too fast (corrected my earlier post with info from the paper).

I note that RED are not currently listed as and authorised ProRes certified company.

Perhaps (more speculation!) - RED are working with Apple to increase the 12bit RGB component of ProRes to be 16bit (ie same as the ProRes Luma) to match the depth that Dragon R3D's can encapsulate?

AJ

Sorry Antony, I think you are wrong
<snip>

 

[Bob Gundu]

Jarred and the killer engineers will have some amazing news soon. :smile:

 

[Phil Holland]

Hi.

 

[Nick Morrison]

WOW!!!!! THEY DID IT!!!!

 

[Nook Kim]

I kind of had this was going to happen.. and it did. Wow..