Acquisition Formats for Scarlet -X

[Jean Simenon]

When you debayer a 5k frame you get the full resolution 5120 x2700 pixel image with every pixel being a fully defined RGB pixel. How much compression you can get away with before the image visually degrades depends on how sophisticated the compression algorithm is at compromising only the data that is not visually relevant.

When you debayer a 5k frame you get "a" full resolution 5120 x2700 pixel image, not the original one if you have compressed it.

 

[Terry VerHaar]

... quite impossible to compress without loss.

...

I really don't believe this to be a true statement.

Jean - maybe it would be easier to explain this to you if we knew what end you were headed toward. I can't decide if you are arguing about RED's approach or about the explanations you are getting. Does David's comment above help?

Like I said before - where are Graeme and Deanan when we need them. LOL

 

[D Fuller]

It is possible to compress without loss, but not very much. Most of Red's codecs are lossy compression. If I remember correctly, 3:1 is mathematically lossless (meaning you get back exactly the same pixels on decompression that you had before compression). From there, every step of compression increases the lossiness of the compression scheme. At about 6:1 or 7:1, you become as lossy as Redcode 42, if I remember correctly (which is pretty darned good). You get to choose how much compression you want to use.

But it matters a great deal what you choose to compress. Try to keep the concept of Raw image separate from the concept of Compression. The recording of the Raw image gives you the possibility of color-grading the image as it is being debayered, which means you can control color temperature and tint in the most basic step of the process. That has profound consequences on the control you have over the image. Recording Raw lets you work with an unsharpened image in post. You can apply the right kind and amount of sharpening in post, rather than dealing with an already-sharpened image that comes out of a camera. All that is true whether the Raw image is compressed or not. What's important is that you are working with the Raw sensor data.

Because the image is unsharpened, there is a much lower tendency to develop the macroblocking that fine, sharp details produce, so the nature of the artifacting is different. And because the image is debayered after compression/decompression, the debayer process masks some of the artifacts you'd expect to see in a compressed RGB image. Finally, if you are working to a delivery that is smaller that 4K, the downscaling that's done to get to 2K or 1920x1080 eliminates much of both noise and artifacting by reducing it in scale so much that it doesn't matter. (That's not an advantage of Raw recording, its an advantage of recording 4K.)

As far as compression goes, recording the Raw image also means that you start with 1/3 of the data you would have if you turned it into an RGB image first. To get to the same data rate with an RGB image, you would have to compress three times as much, causing far more loss to the image. And that's where the downsampling scenario you have been talking about falls apart. In order to downscale the image, it first has to be debayered to RGB. So you start with 3X as much data as the Raw file would have to be. You can then make it smaller than the RAw file if you downscale and compress it enough, but imagine how lossy that process would be!

 

[Gavin Greenwalt]

I can't imagine that you wouldn't be losing anything for color/luminance data. The data is 16-bit information and quite impossible to compress without loss. Beyond that the compression is saying that there are 3 red in row, eliminating the data for 2 pixels...

Very simple: RAW == Unprocessed sensor data. This means it simply captures the sensor data, without assigning color value or real world correlation. What makes it RAW is that since it's RAW sensor data you can adjust white balance, gain or colorspace since none of that since you can characterize sensor data to real-world values. You can't as easily know from a random JPEG what was actually captured by the sensor.

RAW does not necessarily mean Bayer Pattern. If you had an RGB camera you could still capture RAW sensor data for each photo site (and then apply White balance, gain and colorspace interpolation in post.)

All you need to know when someone says "RAW" is that it's unprocessed sensor information. That information might be compressed or not. But it's still RAW. If you shoot 8mm or 35mm a negative is a negative. You compress or don't compress a RAW file is still a RAW file.

A RAW file isn't intrinsically better than a normal file. You could have a camera that processes its images into a super wide XYZ or RGB colorspace that would allow even more flexibility with Gain/Color/Whitebalance but *wasn't* RAW. RAW just has traditionally been better than the alternatives.

As to 3/2 pixels == 3k + 2k. This is a case of the whole being more than the sum of the parts. If you can see the back half of an elephant you can usually infer a pretty accurate assumption that there is a front to the elephant as well. Similarly, you can infer the expected pattern based on surrounding values.

 

[Tyson Perkins]

Sorry for the rookie question....

But using these external recording devices to 1080p, although I'm aware of the qualitative loss of image - do we also lose the framesize, i.e. does it extend the same windowing that the Scarlet does in-camera when shooting 1080p?

 

[Jean Simenon]

I really don't believe this to be a true statement.

Jean - maybe it would be easier to explain this to you if we knew what end you were headed toward. I can't decide if you are arguing about RED's approach or about the explanations you are getting. Does David's comment above help?

Like I said before - where are Graeme and Deanan when we need them. LOL

From what I'm understanding about compression for color is that it's not possible to do it without loss. Not really arguing about RED's approach since they're not telling anyone what they're really doing ("the secret sauce").

David's answer was definitely helpful.

 

[Jean Simenon]

Very simple: RAW == Unprocessed sensor data. This means it simply captures the sensor data, without assigning color value or real world correlation. What makes it RAW is that since it's RAW sensor data you can adjust white balance, gain or colorspace since none of that since you can characterize sensor data to real-world values. You can't as easily know from a random JPEG what was actually captured by the sensor.

RAW does not necessarily mean Bayer Pattern. If you had an RGB camera you could still capture RAW sensor data for each photo site (and then apply White balance, gain and colorspace interpolation in post.)

All you need to know when someone says "RAW" is that it's unprocessed sensor information. That information might be compressed or not. But it's still RAW. If you shoot 8mm or 35mm a negative is a negative. You compress or don't compress a RAW file is still a RAW file.

A RAW file isn't intrinsically better than a normal file. You could have a camera that processes its images into a super wide XYZ or RGB colorspace that would allow even more flexibility with Gain/Color/Whitebalance but *wasn't* RAW. RAW just has traditionally been better than the alternatives.

As to 3/2 pixels == 3k + 2k. This is a case of the whole being more than the sum of the parts. If you can see the back half of an elephant you can usually infer a pretty accurate assumption that there is a front to the elephant as well. Similarly, you can infer the expected pattern based on surrounding values.

It all starts to make sense. But from other comments and from my understanding. A file on your redmag at 3:1 is not going to be able to "debayer" the same way a 18:1 will, correct? You're essentially losing information at 18:1 compared to 3:1.

Then is decoding 18:1 instead of 3:1 a longer process?

 

[Terry VerHaar]

Yes - obviously (I think it's obvious, at least) higher compression means more loss, lower compression means less loss. But, it's not as if the ASICs in the camera or REDCine-X have to figure out how to do this process each time; it's an algorithm built into the firmware. So, technically, I am sure it takes "longer" to decompress 18:1 than it does 3:1, I am not sure it is really an issue of any sort.

Remember, mathematically lossless compression means that the data can be re-created without loss from the original. You are right (AFAIK) that above about 3:1 there is some loss. RED, however will argue that it is "visually lossless" (certainly somewhat subjective) up to some point above that. Of course, at 18:1, I imagine it would be noticeable in at least most footage.

Seems like this discussion all started along the lines of recording a 1080P signal to the SSD. Perhaps at least now you can see that this would require a whole new set of software algorithms and even some additional in-camera hardware to accomplish. And, it doesn't appear to be something RED is terribly interested in pursuing thus far.

 

[Antonio Ribeiro]

Hi folks,

The whole thread makes for very interesting reading (while waiting for things to render on my timeline). I have bought a Scarlet X with quite a few other items so I can function, but I am worried about working with the files on post. That being said, I was the editor of a feature film shot entirely on the Red 1 using my current system (MacPro Quad Core Intel Xeon 3Ghz / 8GB RAM / Nvidea GeForce 8800 GT) and had no problem whatsoever, using Prores HQ at 2K in the first instance and then reconfirming the footage back to the red original files, for the grading in Australia. I didn't even use for this, instead, used the Red plug-in available for FCP 7, when you use the Log and Transfer option.

The issue here is I simply cannot afford a Red Rocket and fear the workflow from the Epic/Scarlet isn't as straight forward. Can somebody hopefully clarify this issue for me?

Thanks.

 

[Antonio Ribeiro]

i meant to say 'I did't even need to use Redcine' on sixth line counting from below.