What Type of Compression do Red Cameras Use? Wavelet?

[Adrian Jebef]

Here is how I compare the RAW images ( Red calls r3d raw, but it really isn't , but for arguments sake Ill go with that)

i shoot with the Red, process r3d in RCX to a 16 bit tiff sequence. Highest quality settings, non clipping color correction.
I then shoot same scene with Canon in CR2 raw format. Process in adobe camera raw to 16 bit tiff at best settings and no clipping in blacks or whites.
Both shoots rack focus slowly to ensure the best focus is captured somewhere in the sequence of frames. On tha canon I would shoot a dozen or so frames. The red gets more frames obviously because it's a movie camera.

I then color grade one to match the other . A curve fit match is enough to match if I'm mainly looking at detail issues. These steps are now in Fusion or nuke, high bit depth.
zoom in 800% of so and look at fine details.
For extra fun, look at the dark shadow details, brighten to see what detail there is visible in each. The red tends to lack detail or even noise at the darkest, almost black areas. The lack of noise is not consistent with the noise in brighter parts of the image.

Have you tried this ?

I plan on repeating this test when I receive my black magic 2.5 K camera. It has no compression and an actively cooled sensor. Should be revealing . I'm guessing Red will up the priority of improving demosaic in RCX , which will be welcomed by many.

I, too, have seen this in EPIC footage. It can be slight but it is there. There also seems to be so much confusion surrounding RED's R3D RAW workflow that most just assume that RAW = Perfect when, in fact, a RAW file is only as perfect as the sensor that created it (and is made even less perfect by compression and processing). As Les alludes to, the sensor technology in a Canon still camera, an Epic, and a host of other manufacturers is really not at all that different. Just because we now name our sensors doesn't automatically make them "revolutionary".

I look forward to seeing some of your test stills, Les.

 

[Les Dittert]

OK, I ran some tests. I used a difficult scene with a lot of details.
Of interest to this here thread, I compressed the raw derived DSLR image with Jpeg 2000 to get to the same size as the R3D snap.
I had to use quality setting 12 ( out of 100 ) to get the file size to match the RC3 red version.

On the images it is plainly obvious that even the old Canon XTi image holds more detail, as it is uncompressed.
RC3 is not lossless, no way no how.

http://reduser.net/forum/showthread...n-test-examples-DSLR-images-thrown-in-as-well

 

[Jeff Kilgroe]

The meaning of these tests are in the eye of the beholder... REDCODE is not lossless, well, it can be, but not on Scarlet. You want "mathematically lossless", as in there is no information lost or left out, then you're shooting EPIC at a compression level of 2.5 : 1.

The next factor to consider is what is RAW and how do different camera companies approach it... Canon RAW is not as RAW as RED RAW... Canon's RAW is raw like commercially packaged beef jerky. Lots of additives and heavily processed. So why call it "RAW"? ...Because they can. They add in sharpening and other detail enhancements, noise reduction, lens compensation, etc.. It's in there along with ISO baked in as it's dependent on sensor gain. Before you jump to conclusions about which camera is actually showing more real detail, you might want to measure it in a quantifiable way -- resolution charts and zone plates. Sure, those aren't real-world photography subjects and most people don't give a rat's ass if a zone plate looks good on a camera, but rather what the actual images they shoot look like. But it's always an interesting experiment... It's often surprising what we actually see vs. what we think we see when it comes to images out of DSLR cameras.

Transcoding to other codecs and doing comparisons just adds more confusion. For comparative purposes, an output to TIFF or similar from these cameras' native codecs is really the only way to go. Cineform RAW isn't really RAW in many cases. I can't comment on its current level of support for Canon RAW and conversion, if it even exists, but in most situations, Cineform builds their "RAW" by deconstructing RGB data back into an approximation.

Optical path -- namely the sensor block and OLPF system are different between the two cameras, with two somewhat different theories at work.

Still camera vs. motion camera... RED hasn't enabled the Stills modes, which will have different (lower) compression offerings, lossless modes, etc..

As for in-camera processing like we see in DSLR's, don't count on seeing that from RED. It gives us more freedom in post and also doesn't bake-in image characteristics we may not like. The XTi crops you have posted do give the appearance of some more detail, but on close scrutiny there's also more edge artifacts and aliasing.

 

[Les Dittert]

Jeff,
The Red Code does unrealistically well when you image most charts, as there are more bits devoted to the high detail areas where the measurements are likely to be taken from. One solution is to surround the chart with harder to compress detail, but as you can guess, that might be difficult to judge how much 'hash' to throw at it.

Regardless, my interest is in the resulting images and what can be achieved with any tools we have at our disposal. I don't care what Canon is doing because what I see coming out of ACR is pretty good.

With regards to transcoding, for my other thread I only used a lossy codec at the end, for the JPG Q=12 examples I posted on the forum, and the r3d IQ deltas I'm showing are much greater than any slight jpg quality loss on final my display. As far as your reference to Cineform, you perhaps are referring to my mention of some Cineform RAW tests I did with r3d's. In that case I actually did it the right way, saving Kakadu j2k decoded RGGB to Cineform RAW. Wasn't easy. Will not be commercially available, it was just an experiment.

I hope Red does add a still mode for slow frame rates. Uncompressed or J2K lossless would be fantastic!
Go ahead and blow up / gamma up those examples I posted. The detail diffs are super obvious. As they should be, with frames that are only 4 megabytes compressed!
As Bob Dylan says, I don't need a weatherman to tell me which way the wind is blowin.


300% blowup, nearest neighbor

 

[Craig Parkes]

What compression ration Redcode did you shoot? Canon looks electronically sharpened (in camera), R3D looks like some compression is going on. Post sharpening and aggressive grade would likely get the R3D closer to the Canon, the Canon's over sharpened look (and increased noise) looks to be harder to get rid of.

 

[Les Dittert]

See the other thread for full details, but: RC3 as it says in the image posted.
Canon had no sharpen in post, nor did it have any noise reduction.
I could have made the Canon look much cleaner with Adobe Camera Raw, but that was not the point here.

At the RC3 setting, the actual compression was more like 4 to 1 . ( the r3d snap was 4.2 megabytes )

With a compression of 4 to 1, there will be detail loss on that frame, as it is a very busy detailed frame.
A shallow focus, or motion blurred frame would behave much better for sure.
Red Code is NOT a lossless compression , even at RC3 or RC2.5 .

Here is the R3D , you can try post sharpening it or whatever and feel free to post the result image here so we can learn.

https://dl.dropbox.com/u/61349012/DSLR_and_Red-compression_test/A005_C001_0929SO.0000000_S000F.R3D
https://dl.dropbox.com/u/61349012/DSLR_and_Red-compression_test/A005_C001_0929SO.0000000_S000F.RMD

What compression ration Redcode did you shoot? Canon looks electronically sharpened (in camera), R3D looks like some compression is going on. Post sharpening and aggressive grade would likely get the R3D closer to the Canon, the Canon's over sharpened look (and increased noise) looks to be harder to get rid of.

 

[John Marchant]

I think a test of the converse situation would quickly allow one to judge where the softness comes from in the Red image (compression or optics). Just shoot a blank out of focus BG with a single pin sharp element in frame (a wire, or piece of thread). I'd lay good money you'll see no sharper fine detail than you're currently seeing in complex scenes, as the OLPF becomes the limiting factor at 3:1.

The OLPF is heavy and simply won't let you have single-pixel sized detail, or you'd be suffering with aliasing.

 

[Björn Benckert]

See the other thread for full details, but: RC3 as it says in the image posted.
Canon had no sharpen in post, nor did it have any noise reduction.
I could have made the Canon look much cleaner with Adobe Camera Raw, but that was not the point here.

At the RC3 setting, the actual compression was more like 4 to 1 . ( the r3d snap was 4.2 megabytes )

With a compression of 4 to 1, there will be detail loss on that frame, as it is a very busy detailed frame.
A shallow focus, or motion blurred frame would behave much better for sure.
Red Code is NOT a lossless compression , even at RC3 or RC2.5 .

Here is the R3D , you can try post sharpening it or whatever and feel free to post the result image here so we can learn.

https://dl.dropbox.com/u/61349012/DSLR_and_Red-compression_test/A005_C001_0929SO.0000000_S000F.R3D
https://dl.dropbox.com/u/61349012/DSLR_and_Red-compression_test/A005_C001_0929SO.0000000_S000F.RMD

Question did you use RED rocket?
The r3d is soft since the camera does not introduce any sharpning, and the same goes for redcine, the image remains soft if you do not apply any sharpening algorithms. This is a good thing. It makes it possible to post the image and then pull sharpness. Things like compositing and grading should go underneat the sharpening process. Canon delivers a ready to use intermediate so even thier soft camera setting is actually introducing digital sharpening.

If you do not use red rocket your image will be a tad less sharp at 1:1 resolution compaired to the rocket render that has a slight differnce in the debayring... the rocket has some sort of hardware sharpning that the software render is missing out on... We did a bunch of test and came to the conclution that if you are going to use the material att full ress of the camera.. i.e 5k for epic then you need the rocket to get the best result. However if you down interpolate to HD or such than the software render is providing a just as good or even better interpolation.

 

[Gabe Conroy]

OK, I ran some tests. I used a difficult scene with a lot of details.
Of interest to this here thread, I compressed the raw derived DSLR image with Jpeg 2000 to get to the same size as the R3D snap.
I had to use quality setting 12 ( out of 100 ) to get the file size to match the RC3 red version.

Wait, so you compressed an RGB image to JPEG2000 to get it to the same size as a pre-debayered image which is going to start out being a *third* of the size of an RGB image of the same resolution...

 

[Les Dittert]

Wait, so you compressed an RGB image to JPEG2000 to get it to the same size as a pre-debayered image which is going to start out being a *third* of the size of an RGB image of the same resolution...

Yes, I compressed an RGB image, and clearly stated that it is not a direct comparison with a CFA RGGB compression.
Still an interesting data point , Q=12. Sharpness still higher than the RC3 version, starting to point to OLPF softening reasons.

Re:Red Rocket. I am very aware of the RR difference. The Software version should have that algorithm as a choice for sure. That should take like three days to code up.

@John: Next time I do the test I will shoot the scene two ways to test this: full frame, and then with a cardboard with a hole punched in it in front of the lens. This way most of the frame would be blurred and easy to compress, except for the center. This will, as you said, determine if it's the OLPF doing the extra softening. How does that sound ?

I'm repeating these tests as soon as I get the Black Magic cam. I will be comparing four cameras next: BM, Red Epic and Scarlet, Alexa, Canon DSLR.
This test will be on a stage here in LA.

 

[Craig Parkes]

See the other thread for full details, but: RC3 as it says in the image posted.
Canon had no sharpen in post, nor did it have any noise reduction.
https://dl.dropbox.com/u/61349012/DSLR_and_Red-compression_test/A005_C001_0929SO.0000000_S000F.R3D
https://dl.dropbox.com/u/61349012/DSLR_and_Red-compression_test/A005_C001_0929SO.0000000_S000F.RMD

RC3 I take to mean Red Color 3, which is color space, not Redcode 3:1 Compression ratio. I see from the clip that it's 3:1 compression so good to know we are starting at the highest quality compression point Red Offers. Will have a play and post back...

Ok, have had a play and came up with this:

http://www.afterglow.co.nz/r3d/A005_C001_0929SO.jpg

Now - there are a lot of caveats here:
Firstly, I have added noise and further post sharpening and edge sharpening to help increased perceived sharpness. This is a more shotgun approach than whatever is causing the weird blocky colors going on in the Canon example, which is obviously part of either their raw processing or recording process, but it does get us (in my opinion) closer to the Canon T2i footage in terms of perceived sharpness.

Does it do so with way less perceptible aliasing? Absolutely.

Can I get the edges as definable as the Canon T2i - no

Is it likely that the differences are entirely processing/compression related - no

Is it likely that the differences are optical, as per the comments in the other thread, caused by an OLPF designed to avoid aliasing at any cost? I think so.

Is it likely that it's the significant difference is due to R3D compression at 3:1 rather than an optical difference? I dont think so, because I see more obvious COMPRESSION/PROCESSING artifacting (weird blotchy colours that shouldn't be there on the bark) in the Canon T2i footage then I do in the R3D file, which speaks to me that there is more going on on the Canon side then the R3D side electronically, because the Canon is making up data in some way to get those edges as noticeable as they are.

Can the Scarlet match the T2i in terms of sharpness with no additional artifacting? No.

Can you monkey around with the R3D to get it looking closer to the T2i (especially when looking at it at 100% rather than at a 300% crop?) - I think so. It takes some tricks and possibly isn't worth the effort.

At 3:1 I wouldn't blame the compression as much as the fact that you are shooting with two different cameras.
The T2i's processing is creating weird blotches, and aliasing.
The added noise on my manipulated R3D output and sharpening creates film grain like color artifacts, and isn't as sharp at the Canon T2i.
The unsharpened R3D is noticeably softer again but has no discernible color artifacting.

Even applying compression to your Canon t2i won't get rid of it's color artifacts, sharpening the R3D won't get it as sharp as the Canon T2i - which is more important probably REALLY depends on your application, but I prefer R3D's way of doing things for the most part - oh, and it can shoot at a much faster frame rate.

 

[Les Dittert]

Thanks for taking a look.
I think the Canon can be made far better looking by adding some processing to it as well.
If I have time, I'll play with it some more, but it's a dead end anyway, because it's a still camera ( except for time-lapse ) !

More interesting: I shot a test chart with the Red with it's OLPF removed. Back focus way off so I could only get the chart in focus rather close.
I was expecting more aliasing on the line pairs, but it is really not that bad.
The bonus: I'm getting 'canon like' sharpness. Shooting charts on a Red is a crap shoot because the compression is not stressed much, so I'll withhold my elation until I get outside and shoot some high detail compression buster type textures.
I fear that the next level of image detail loss might show up: Smoothing due to high frequency suppression by the compression. I suspect that the compression will be overwhelmed by the detail.
A card with a hole to restrict most detail but the center will be used to see if that's the case.

A still image rendered with no resize : https://dl.dropbox.com/u/61349012/no_aa_red_tests/4k_noaa.00.jpg

 

[Les Dittert]

Just to see how far the Canon T2i image can be pushed, here is another version. Less noise, more detail.