EPIC 12-bit and 16-bit Recording Options?

[D Fuller]

I'm pretty sure that it does not record 12 bits per sample because it would not be able to capture 13.5 stops. The minimum would be 14-bits.

Not so true. Bit depth is only relevant to the "resolution" of color depth, not dynamic range. 20 stops could be mapped to 10 bits, or 6 bits, for that matter. You'd just see more banding.

 

[Pawel Achtel]

Not so true. Bit depth is only relevant to the "resolution" of color depth, not dynamic range. 20 stops could be mapped to 10 bits, or 6 bits, for that matter. You'd just see more banding.

True, but then, as you say, you would need to apply a mapping, which we don't as we are recording RAW. So either the camera has 13.5 stops of DR or it records 12-bit, but can't be both. If it was RGB, I would agree with you, but not when it is RAW.

 

[Patrick Tresch]

From RED ONE operation guide build 30.7.0 p. 23

IMAGE PROCESSING
The digital images received from the MYSTERIUM or MYSTERIUM X sensors are formatted as pixel defect
corrected (but not color processed) RAW data - similar to digital RAW stills cameras.
The sequence of RAW images received from the sensor is compressed using proprietary 12-bit REDCODE
RAW compression. The REDCODE RAW data recorded is independent of the RGB signal monitored
in the monitoring path. ISO, White Balance and other RGB color adjustments made to the RGB
monitoring path are not burned into the recorded 12-bit REDCODE RAW data.

Pat

 

[M Most]

True, but then, as you say, you would need to apply a mapping, which we don't as we are recording RAW. So either the camera has 13.5 stops of DR or it records 12-bit, but can't be both. If it was RGB, I would agree with you, but not when it is RAW.

Any image sensor is in fact an analog device. Analog devices don't have bit depth. The bit depth comes into play when the information is digitized, which in an image sensor either happens on the imaging chip or external to it, but it always happens. The dynamic range of the sensor and the bit depth of the information are two completely different things. The bit depth represents the precision of the sampling, not the range of values it can describe.

 

[D Fuller]

Any image sensor is in fact an analog device. Analog devices don't have bit depth. The bit depth comes into play when the information is digitized, which in an image sensor either happens on the imaging chip or external to it, but it always happens. The dynamic range of the sensor and the bit depth of the information are two completely different things. The bit depth represents the precision of the sampling, not the range of values it can describe.

Exactly what I meant. But you said it better. Thanks Mike.

 

[D Fuller]

From RED ONE operation guide build 30.7.0 p. 23

IMAGE PROCESSING
digital images received from the MYSTERIUM or MYSTERIUM X sensors are formatted as pixel defect
corrected (but not color processed) RAW data - similar to digital RAW stills cameras.
The sequence of RAW images received from the sensor is compressed using proprietary 12-bit REDCODE
RAW compression. The REDCODE RAW data recorded is independent of the RGB signal monitored
in the monitoring path. ISO, White Balance and other RGB color adjustments made to the RGB
monitoring path are not burned into the recorded 12-bit REDCODE RAW data.

Pat

The Epic manual is far more unclear about it. This is from the Epic manual:

"The digital image received from the sensor is formatted as a pixel defect corrected (but in all other aspects unprocessed) 12-bit, 14-bit or 16-bit per pixel RAW data frame. When operating in STILLS mode, each image received from the sensor is treated as a unique FRAME; however when operating in MOVIE mode, successive images received from the sensor are collected together into a CLIP. In all other regards, a frame of information gathered in STILLS mode or MOVIE mode is identical."

I have no idea what the criteria might be for controlling bit depth. My suspicion is that it is part of the choice of Redcode data rate, but I have no other information than the manual.

 

[Pawel Achtel]

From RED ONE operation guide build 30.7.0 p. 23

IMAGE PROCESSING
The digital images received from the MYSTERIUM or MYSTERIUM X sensors are formatted as pixel defect
corrected (but not color processed) RAW data - similar to digital RAW stills cameras.
The sequence of RAW images received from the sensor is compressed using proprietary 12-bit REDCODE
RAW compression. The REDCODE RAW data recorded is independent of the RGB signal monitored
in the monitoring path. ISO, White Balance and other RGB color adjustments made to the RGB
monitoring path are not burned into the recorded 12-bit REDCODE RAW data.

Pat

I understand what you are saying Mike and David.

So, do they compress/decompress 14 or 16 bits into 12 or is it 12-bit quantization? I think it must be the former otherwise, if they claim 13.5 DR, some bits would need to be thrown out. Which bits would you throw out? Top and bottom? then you end up with less than 12 bits DR. Middle? Then you are introducing banding in the mid-tones.

I think they are performing at least 14-bit quantization on the analog signal and compressing it somehow into 12 bits, which is then decompressed back to at least 14 bits. I do not think RAW quantization is 12-bits.

 

[David Rasberry]

Isn't the initial quantization bit depth user choice? Compression of the data after quantization is not the same thing as reducing bit depth. I can see perhaps tying initial quantization bit depth to higher compression rates as a method of reducing data files size before compression or to increase frame rates.

 

[D Fuller]

I understand what you are saying Mike and David.

So, do they compress/decompress 14 or 16 bits into 12 or is it 12-bit quantization? I think it must be the former otherwise, if they claim 13.5 DR, some bits would need to be thrown out. Which bits would you throw out? Top and bottom? then you end up with less than 12 bits DR. Middle? Then you are introducing banding in the mid-tones.

I think they are performing at least 14-bit quantization on the analog signal and compressing it somehow into 12 bits, which is then decompressed back to at least 14 bits. I do not think RAW quantization is 12-bits.

No bits need to be thrown out to record 12-bit RAW. You assign each bit 1/12th of the DR when you digitize. It does not have to be a unit equal to a stop. To help understand this, imagine that the recording was in 32-bit. There would still be only 13.5 stops of dynamic range, but there would be a great deal more precision in the units used to describe it.

EDIT: a 12-bit recording uses 4096 steps per pixel. That would be 303 steps per stop - for each of the colors of the bayer pattern.

But the EPIC manual speaks about 12, 14, or 16-bit data frame. Does anyone have any idea under what circumstances those data rates are employed? or if they are user-controllable at all?

 

[D Fuller]

Isn't the initial quantization bit depth user choice? Compression of the data after quantization is not the same thing as reducing bit depth. I can see perhaps tying initial quantization bit depth to higher compression rates as a method of reducing data files size before compression or to increase frame rates.

David, that's exactly the question: Is it user choice? Nothing in the menus of the Epic seems to address this (or at least I haven't found it) so it may be an unfinished feature, or it may be a setting inherent in the Redcode choice. Or it may be something else that I don't know about. I suspect the Epic of having higher bit rates because B&W out of the camera looks so damn good.

 

[Pawel Achtel]

...EDIT: a 12-bit recording uses 4096 steps per pixel. That would be 303 steps per stop - for each of the colors of the bayer pattern.
...

I understand this, but I don't think that's what they do. I think they compress 14 or 16 bits into 12 bits, but not on a pixel level but on a group of pixels. This way they do not need to throw away the informaion, which is precisely what would happen if you fit the entire DR into 12-bits. If, as you say, they used 303 steps per stop, it would be equivalent of reducing the actual contrast of the scene, which I'm pretty sure they do not do.

 

[Uli Plank]

If, as you say, they used 303 steps per stop, it would be equivalent of reducing the actual contrast of the scene, which I'm pretty sure they do not do.

Sorry, Pawel, but like many folks in this thread you just don't get the point!
Bits in a digitized format are just a form of resolution – that's why it's called bit "depth". You can squeeze 14 stops into 8 bit (and risk quite some banding) or into 32 bit and have no banding at all, but waste space and bandwidth. But you don't loose contrast by doing that. RED is obviously – judging from image quality and needs of recording media – making a very clever compromise here. Nevertheless, just like the OP, I'd like to know what they are actually doing and if you can influence it. The manual is contradictory about this.

In some other cameras log curves would come into play here, redistributing the bits in a non-linear fashion, since human vision is nonlinear too. That's just another form of perceptual compression: you shuffle bits from the highlights, where our eyes don't differentiate very well, to the shadows, were we are perceiving very small differences.

You can squeeze 13.5 stops into 10 bits log pretty well, but it's still a form of compression.

 

[Gunleik Groven]

Thing is that RED early on made a rather strong argument for linear mapping.
With some redistribution of precission between the last bit and the lower exposures.

Nothing has been said or seen, that makes me think that philosophy is radically altered. Actually I think the extra precission headroom of 16 bits is used both for DR and processing a lot on the Epic.

 

[Pawel Achtel]

Uli, I understand perfectly what you are saying. What I am suggesting is that REDCODE does not use simple 12 bit per pixel sampling. The manual says something different: they use 12-bit compression. This does not mean they use 12 bits per colour sample. There is a difference.

 

[Patrick Tresch]

All this is getting interesting.

I also think 12 14 16 could be linked to some RC choices. Greame has surely the answer but as their internal cooking is a sensitive topic, I don't know if he will give us some answers.

It is evident that MX on the Epic looks way better than on the RED ONE.

Jim said it was due to improoved electronics. They surely also improoved other parts of the system.

Pat

 

[David Rasberry]

A straight linear quantization would not yield equal bit samples per stop though. A stop is a logarithmic value: 20log*(2)

 

[Gunleik Groven]

A straight linear quantization would not yield equal bit samples per stop though. A stop is a logarithmic value: 20log*(2)

Nope, that's true and the whole point of Graemes "borrowing" on the R1
Gives lower precission in the lowlights and higher in the highlights.
Thus the "borrowing" from the top to the bottom. But it's not done over the full range, thus still in principle linear.

 

[Uli Plank]

The manual says something different: they use 12-bit compression. This does not mean they use 12 bits per colour sample.

There are no color samples at that point – files are encoded without de-bayering.

Very interesting thread, nevertheless. Such arguments come up in discussions where fans of other camera systems are quoting such figures. It would be great to get some more facts from RED, but if they do something proprietary here, I can understand their silence.

 

[David Rasberry]

Whatever Graeme's doing with his magic pixels it is marvelous. I can't find the frame now, but I was pixel peeping a full 5k res 16 bit tiff of a hand and arm in bright sunlight. As the values rolled off smoothly into the deeper shadows, fine detail rolled off too so the deep shadows were massed and blended in a way similar to what a painter would do. No pixel blocking or banding anything like that, just a nice smooth continuous falloff in both value and detail at the extremes. Never seen anything like this from any digital camera. Even zoomed way in you could not discern individual pixels at all, the image had a very smooth organic look to it.

 

[Robin Balas]

Sorry, Pawel, but like many folks in this thread you just don't get the point!
Bits in a digitized format are just a form of resolution – that's why it's called bit "depth". You can squeeze 14 stops into 8 bit (and risk quite some banding) or into 32 bit and have no banding at all, but waste space and bandwidth. But you don't loose contrast by doing that. RED is obviously – judging from image quality and needs of recording media – making a very clever compromise here. Nevertheless, just like the OP, I'd like to know what they are actually doing and if you can influence it. The manual is contradictory about this.

In some other cameras log curves would come into play here, redistributing the bits in a non-linear fashion, since human vision is nonlinear too. That's just another form of perceptual compression: you shuffle bits from the highlights, where our eyes don't differentiate very well, to the shadows, were we are perceiving very small differences.

You can squeeze 13.5 stops into 10 bits log pretty well, but it's still a form of compression.

First I need to point out that I am talking real life implementation here and not theoretical math, and I am an engineer - not a mathematician or physics expert.

If you are squeezing 14 stops into 8 bits RAW lin - how many levels do You expect there are to describe the contents of the darkest stop's worth of info (remember the LINEAR state)?
There are none, nothing can change that. I suggest reading up on binary arithmetics if this sounds strange.

If one assumes a log encoded RAW file you may squeeze in more stops than bits, but to go from lin to log, in the digital domain, the darkest stop contains the least resolution mathematically speaking. The lowest bit only describes 1 level of info and the highest describes 2^n-1 levels, which is 128 for a 8 bit file, hence a shadow issue and not a highlight issue. Regardless of redistributing bits etc. there are no info left to distinguish the contents or levels in the shadow stops when doing the lin to log transformation. But in theory it is possible to do as you describe if you accept the extreme banding in the shadows which is in my opinion not a realistic assumption in the real world.

To properly provide a gradation in the darkest shadows you really need more bits than stops when doing the binary math step from lin to log.
The damage is irreversibly done when the digitizing of the analog signal into the digital linear representation occurs. This is normally step 1 in the A/D chain.

In a 8-bit lin to log transformation the top bit is describing the brightest values in the captured light, it causes the (from bright to dark) 1.stop to contain 128-levels, 2. stop 64-levels, 3. stop 32-levels, 4. stop 16-levels, 5. stop 8-levels, 6. stop 4-levels, 7. stop 2-levels and 8. stop 1-level. There are no definition to describe 14 discreet stops of information with a 8-bit lin file. It would look cr#p in the shadows.

To avoid all of this one needs to go from a lin signal to a log one in the analoge domain prior to digitizing, then you need a lot less bits to keep all info. Hence all this is not a sensor issue but a A/D digitizing issue. The sensor might capture 13.5 stops, but you loose the ability to distinguish those dark gradations

I am not aware of any digital image sensor which have log characteristics in the analog domain. There might be one around which isn't described in the literature tough.

IMHO 16-bits is a very good thing if you wan't to capture 13.5 stops of linear data that needs to go through a lin-log transformation to be usefull. It gives a realistic shadow representation for digital images both stills and moving. Anything else assumes you are in a log domain prior to digitizing.

Its the way it have been working in A/D converters and is the way it is working today. Maybe someone can think outside the box and come up with a whole new set of thinking to accomplish a direct analog to digital log quantization method, and you are then likely a nobel candidate if you do.

I wonder if I got the message through, but the whole point is its about the stuff between black and white not the absolute ends describing black and white - unless you are producing a lithographic look.

I appoligize for the awkwardly arranged line of arguments - its a bit late and I started out in the wrong end with all this.
But please prove me wrong, and the engineering world will be greatfull for overcomming one of the major challenges in A/D conversions.