REDCode from a technical standpoint

[AndreeMarkefors]

Can someone explain the capabilities of REDCode?

It gets labeled as RAW, yet it is encoded.

I've heard about a lossless version, that to my mind could fairly be labeled as RAW, since all the sensor data is intact, but what about the lossy versions?

I realize there's a little thing called "reality", and I think REDCode is awesome. I'm not knocking it in any way—but I do want to understand.

If using a lossy version, like 12:1, how does the headroom compare to if it had been a 100Mbit h.264. At what bit depth does 12:1 get recorded?

I know and understand active meta data, but I want to wrap my head around this on a technical level.

Please help!

 

[Dan Hudgins]

What's RAW in RAW

What's RAW in RAW

WHat is RAW in REDCODE as far as can be made out is that it is not white balanced, that gives you the headroom above green clip in the red and blue for the DRX feature to work, see it in the de-Bayer menus.

REDCODE seems unique in that rather than using high resolution Luma and lower resolution Chroma like normal cameras, it uses a 1:1 interpolated green image perhaps, and it seems two smaller red and blue images that are not interpolated before compression, but rather enlarged to full size after de-compression. You can see if you look at underexposed shots in 3 color seperation that the green image looks about twice as sharp as the red and blue images and the noise also looks smaller in the green image.

How much the JPEG2000 like compression affects the three black and white images for the Red/Blue and Green depends on the subject matter, early on if the camera was pointed at tree branches it would sometimes report "codec error" or something like that because there was too much detail to encode at once, but they have made many improvements to their compression and de-compression and you can look at the results of that work.

I did ask about if they were going to bypass the compression for their lossless mode in epic which is going to have about 1:2.5 compression, not be fully uncompressed, and it seems it will not bypass the compression alltogether but rathere adjust the compression encoding to some level that is said to be "lossless". Although without a True RAW port, you cannot have two copys of the sensor data to tell if the de-compressed data is an exact match to the 14th bit with the original sensor data, not that that would matter much in what you see, maybe.

If they are using log encoding of the 14bit data into 12bits, then there would be scaling errors anyway.

 

[AndreeMarkefors]

If they are using log encoding of the 14bit data into 12bits, then there would be scaling errors anyway.

Thanks Dan.

Is the bit depth precision for REDCode 12 bit at all compression ratios?

 

[Stuart English]

RAW

RAW

Yes, all REDCODE RAW recordings are at least 12-bit depth.

 

[Keith Putnam]

It gets labeled as RAW, yet it is encoded.

"RAW" simply means that the image data has not been permanently debayered, not that it is uncompressed.

 

[David Rasberry]

Redcode is not an RGB or YCbCr encoded format like video. Video records data as a preprocessed and predefined image of RGB pixels.
Redcode compresses Bayer raw data to an otherwise unprocessed file where the 12 bit data for each R pixel, B pixel and G pixel is recorded in a manner that allows it to be post processed in the same way a raw uncompressed data file from any Bayer camera can be post processed. It is the raw sensor data that is recorded to the file and that data is only affected by how the sensor was exposed. Pixel values are recorded in response to light levels, aperture settings, and exposure times as defined by frame rate and shutter speed. Everything else is meta data recorded separately and applied to the raw file non-destructively either in camera or in post.
Video camera settings are recorded in the encoded image and can't be directly changed in post. Key values that we think of as predefined in a video recording include white balance, gamma, gain, pedestal, and various knee and toe compression schemes to keep the recorded data within the defined limits of the video format.
None of these are a permanent part of a Redcode raw file.

 

[David Rasberry]

Yes, all REDCODE RAW recordings are at least 12-bit depth.

Will Scarlet have a 16-bit option like Epic-X or will it be limited to 12 bit?

 

[Dan Hudgins]

In the same way?

In the same way?

Redcode compresses Bayer raw data to an otherwise unprocessed file where the 12 bit data for each R pixel, B pixel and G pixel is recorded in a manner that allows it to be post processed in the same way a raw uncompressed data file from any Bayer camera can be post processed.

I don't think that is an accurate statement, as far as I know, REDCODE can only be converted into RGB output by the RED programs and their SDK code, none of which is open to see what is going on.

Bayer type DNG files (not Linear DNG which are a TIF file without white balance more or less) cannot be made from REDCODE other than by "reverse de-Bayer" from an RGB file.

Likewise to make a Bayer DNG from the wavelet compressed data you would need to pick pixels in the three RGB images that have had their brightness altered by the compression, so in fact it would end up being a sort of reverse de-Bayer from a non-white balanced RGB image, like making a Bayer DNG from a Linear DNG, not from true lossless delta encoded Bayer data where you get back 100% of the original data and an prove that with the console FC command.

REDCODE is a lossy compressed format as it has been used in the RED ONE.

As to if their LOSS-LESS version for Epic is actually going to be 100% lossless to the 14th bit of original sensor data as it comes out of the ADC, cannot be known, since there is no RAW port on the camera to make a reference file of True RAW data to use FC with to find out.

As I noted above using a log LUT in the compression path would make the sensor data fail the FC test for 100% losslessness. Not that it matters how lossy it is too much as with good exposure even very lossy cameras can produce usable results.

Having True RAW sensor data helps avoid some double compression losses, as re-compressing a compressed image does more than just double the losses.

Its best not to confuse people by saying that in most uses REDCODE is lossless without putting the visually in front, as for the way it has been used it is not 100% lossless, it is said to be "visually lossless" and if you shorten that you get to a point that you are implying something that becomes misleading, like shortening "gold plate ring" to just "gold ring" its the "plate" part that is important for the buyer to understand just as much as the "gold" part.

I'd like to see RED sensor data be able to be processed "in the same way" as uncompressed RAW data from other cameras, but you cannot process True RAW data from other cameras in REDCINE-X and you cannot process REDCODE in programs that do not use the RED SDK, right?

 

[David Rasberry]

In the sense that you can't export Redcode Bayer raw frames directly into another raw processor like Light Room or Bibble (yet) you are right Dan.
But in the sense that you have significant freedom to define the deBayer conversion of raw data to a number of different encoded formats with complete control over parameters that would be fixed at recording in RGB or YCbCr encoded file formats, Redcode is as raw as any frame you would get from a digital still camera.

 

[AndreeMarkefors]

"RAW" simply means that the image data has not been permanently debayered, not that it is uncompressed.

No. RAW means "RAW sensor data", or actual sensor data at the time of capture.

If RED tries to redefine RAW (I'm not implying this), we have a problem.

Compression is not a problem if it is lossless.

I feel that Dan understands my question and answers it well.

 

[David Rasberry]

Canon and Nikon compress their raw files too.
For the new Epics, the minimum 3:1 compression has been described as mathematically lossless, meaning that the full 12 bit uncompressed data for every photosite can be recovered.

Higher compression ratios are lossy in the sense that if there are areas of the image where multiple photosites register the same uncompressed value, the compression algorithm would ideally store that value only once with a matrix reference table to replace the photosite values on deBayer reducing the size of the file, but maintaining the full visual quality. This is what would be described as visually lossless.

Visually lossy compression levels would average a narrow range of similar photosite values into one value with a reference matrix at the cost of some fine degree of image quality, but if you are going from a 12 bit per photosite value range to an encoded format at 10 bits or 8 bits per RGB pixel the loss would still not be visible until you reach very high compression levels.

But the differentiating character of Bayer raw is that 12 bit values for each photosite are stored separately in the recorded file without further processing vs a single processed encoded RGB pixel value for a constructed image being recorded. That is the key difference between raw and any encoded format like JPEG or MPEG , or TIFF or sRGB or any other type of encoded image. With Bayer raw the image has to be "developed" from the photosite data.

 

[Graeme Nattress]

REDCODE is compressed RAW sensor data. You can leave RAW uncompressed and un-altered in any way, but this is very rare, and painful to use as you have to do offset/gain/pixel correction in post, and the data for that has to travel with the files from that camera. I think some medium format backs used to work that way.

Some people, like Nikon, give an option to compress raw by log encoding and reducing bit depth. That's pretty simple compression, gives low ratio compression.

We took the other approach, keep the bit depth and use wavelet based techniques to reduce the data rate considerably, while keeping the data in the Bayer pattern format, with no demosaicing or colour matrix.

We are not re-defining RAW. We are compressing the RAW data as compared to a traditional video camera where the data is demosaiced and matrixed and gamma curve applied and then compressed, most likely with a YCbCr transform.

Compression is not a problem if it meets it's goals, and those goals are compressing enough to make the file size and data rate manageable while retaining image quality, dynamic range and post-manipulation flexibility.

Graeme

 

[AndreeMarkefors]

REDCODE is...

Thanks a lot.

Are you familiar with Cineform and First Light?

There's a function there that lets you apply different debayer algorithms depending on need.

Is that/could that be possible with REDCode?

I'm thinking about various levels of sharpening and such...

 

[Graeme Nattress]

Look in REDCine-X - there's a few choices on demosaic - full, half-fast, half-nice. And Full has options for chroma denoise / de-moire and sharpness, although I recommend only sharpening a graded movie rather than sharpening in demosaic as you should sharpen with output device in mind.

Graeme

 

[AndreeMarkefors]

Great, I'll have a look!

 

[Dan Hudgins]

ISO

ISO

One thing to keep in mind is that at higher ISO, most of the image is being taken from the lower 4 to 8 bits of the 12bit data. So the impact of any few bit "smoothing" of the pixel to pixel data may show up more at higher ISOs.

In other words the curve gain that lifts midtone to the higher ISO, also spreads any bit to bit smoothing, and when that spread makes gaps above the 8th bit you can see it as a sort of histogram gap.

I assume that the wavelets do not plateau the tones, but rather make a sort of tonal interpolation, but that interpolation needs to be done at a higher bit depth if linear data is processed, or the interpolation would be in the lower bits and steps could evolve?

I know with un-smoothed data temporial noise reduction (and area filters) can give tones between the ADC steps, but in smoothed data there is no noise around the average so you should end up with tone banding when the ISO "gain" from the curve used to lift mid-tone spreads the tones?

So if you smooth the bottom 4 bits, in a well exposed image the impact would be less than if you push the ISO four stops?