Epic Compression Test Results

[Deanan]

OLPF is made for 5K, system is tuned to 5K.

OLPF is tuned to the pixel size rather than the resolution so you'd essentially pick the same olpf if you had a 2k sensor also.
One could choose a weaker olpf to get more apparent resolution but at the trade off of more aliasing and moire.

 

[D Fuller]

Haha, well even "done right" is going to differ from person to person. Do you want downloadable 1080p prores files, or straight up R3Ds? Personally I feel like it would be rather useful to see what the extreme levels of compression do in a "final output" sort of scenario, but if you want to mess with RAW files I'm happy to post those instead. Just might need some help with a server if that's the case.

My plan would be to set up both an indoor lit scene and an outdoor high-detail natural light scene, with a model and subtle movement; record one take at each of the various levels of compression and share the results. Obviously compression works in all kinds of different ways; this isn't going to scratch the surface of all of the different ways one could shoot. Shooting extreme motion from a vehicle mount, for example, would probably really stress it compared to a locked-off shot. This "test" would be a more "normal" (if that exists) type of shot that might be applicable to narrative work.

Thoughts appreciated as I'll set this up for tomorrow.

A few thoughts:

- One of the most difficult subjects for compression is a moderately wide shot, deep focus, of a scene containing lots of leaves with a bit of a breeze blowing. All those little objects moving randomly puts huge stress on a compression algorithm. (Doesn't work if you use shallow DOF. Then they're just soft color.)
- Extreme motion form a vehicle mount would add motion blur that would actually make the compression task easier.
- Faces close up and skin are not really much of a challenge for compression. (They're great tests for other things, but skin is pretty easy for compression to deal with.)
- Hair, fur, etc. are good tests. Lots of fine edge detail.
- Dark detail should be harder than light-toned detail. There are fewer bits in the shadows, so any lossy compression should show up there especially. And if the shadows are noisy, that'll be another challenge for the codec.

It'll be interesting to see what you find out.!

 

[Zeb B]

Great stuff David. I'm starting an offline list of all suggestions posted here. Please keep them coming. Later we can split up the work to maybe a dozen different test volunteers with a set of procedures and standards to run the agreed on scenarios and post the results. Maybe post to a private Vimeo account for approved owners only. Worry about that later - for now keep the input and progress moving forward. This is important work and is dynamic since each firmware build has the potential of creating new opportunities for improvement and understanding.

 

[Patrick Tresch]

To do such a test a bit scientific we should firstly have to know how the wavelet compression works to try to break it ;-)

Compare different situations with the same setup like:

- rapid movements (ventilator?...)
- high iso settings (to see noise handling when pushed)
- big even surfaces (skys, white wall, green screen, blue screen and even red screen) to see the handling of tiny R & G & B variations
- Green/Bue/Red screen with smoke/hair/rapid movements.
- "high frequencies" situation (leaves in the windy tree, on a bright day)
- Skin and faces (handling of subtile color variations)

What do you think?

Pat

Edit : Sorry didn't see the very usefull answers before posting... ;-)

Thanks D Fuller and Gavin

 

[Gavin Greenwalt]

Rapid movement will only help wavelet compression. It's intra frame not inter-frame so every frame is compressed individually. What rapid movement will do is blur the image and low detail compresses really easily compared to noise.

The hardest thing to compress is random noise (fur, sand, pores, grass etc).

 

[David Battistella]

MYTH = BUSTED

Thanks Deanan...

David

OLPF is tuned to the pixel size rather than the resolution so you'd essentially pick the same olpf if you had a 2k sensor also.
One could choose a weaker olpf to get more apparent resolution but at the trade off of more aliasing and moire.

 

[Brandon J.F.]

Hardly scientific, but Jim Hare shot some comparative shots that we were able to pixel peep at our Redusers meeting last week. To cut the long story short, if you are going to HD delivery, RED RAW 15:1 at 5k is as clean as it gets. You can't see any noise. You can see the noise at full 5k right until about 6:1 and it becomes academic for lower compression ratios.

The compression noise is most evident in shadows. A difficult scene with lots of shadows and detail (cat's fur) was chosen. ISO 800.

If you shoot outdoors or good lighting...you can get away with much higher compression before you can see compression artefacts.

Very interesting. I bought the FXPHD Epic on Location course and in this they say they like using 12:1 compression and it works well even for keying.

What would truly be an interesting test is a blind test where people guess which compression is used for each shot/clip before the compression ratio is revealed.

 

[Hrvoje Simic]

Not all compressions have the same effect.
In this case it doesn't have to be a picture detail artifact, or visible in raw material, even on pixel level. It can a be a color nuance thing, or a material "thickness" thing.
Experienced colorist feels how each material "breaths", and he may notice the difference between 12:1 and 6:1 even if the shooter doesn't.

 

[Zeb B]

Not all compressions have the same effect.
In this case it doesn't have to be a picture detail artifact, or visible in raw material, even on pixel level. It can a be a color nuance thing, or a material "thickness" thing.
Experienced colorist feels how each material "breaths", and he may notice the difference between 12:1 and 6:1 even if the shooter doesn't.

Would love to have a Private Forum where shooters and postwork stake holders can post known issues like this. Wisdom is the fuck-ups you remember or know about. We can all benefit from others raw experiences if there is a private place to share. Maybe Jarred can create another level of Forum that has a higher barrier to entry like invitation only voted on by Mods. Entry would require an NDA preventing reposting raw data or quotes outside of the Forum. Just some thoughts . . .

 

[Bill Anderson]

"Private forum, private vimeo account for approved owners only, private place, another level of forum, higher barrier to entry, invitation only, entry would require NDA to prevent reposts (sharing knowledge)": you are quite a charmer Beadle.

 

[Jean Simenon]

Would love to have a Private Forum where shooters and postwork stake holders can post known issues like this. Wisdom is the fuck-ups you remember or know about. We can all benefit from others raw experiences if there is a private place to share. Maybe Jarred can create another level of Forum that has a higher barrier to entry like invitation only voted on by Mods. Entry would require an NDA preventing reposting raw data or quotes outside of the Forum. Just some thoughts . . .

That's the Burning Man spirit. Get rid of those rats sniffing around the professional's backs! Who are they anyways. I can't imagine that it would be in the interests of RED. They are opening their product to a larger market, and they'll be plenty of people who have good stories, skills and creativity to offer a more competitive marketplace of filmmakers, not just the ones who can afford $100K cameras. Let them come in! I can't wait.

------------------------------------------------------

So we were talking about compression compromises. What do things look like at 13:1? We'd love to know. It also goes to compress to 18:1... why did RED add this kind of level of compression, what does it look like, and what can you do with it?

 

[Patrick Tresch]

Would love to have a Private Forum where shooters and postwork stake holders can post known issues like this. Wisdom is the fuck-ups you remember or know about. We can all benefit from others raw experiences if there is a private place to share. Maybe Jarred can create another level of Forum that has a higher barrier to entry like invitation only voted on by Mods. Entry would require an NDA preventing reposting raw data or quotes outside of the Forum. Just some thoughts . . .

And you would be one of those VIP of course?

I was just wondering if what we see on the LCD is before or after R3D compression?...like before the recording head, pre-read or post-read?

Pat

 

[James Brundige]

With films like Spiderman and Pirates running around, I don't think we have to worry about what gets posted here. Clearly the camera can deliver at the highest level of quality. The DP reports Spiderman shot at 5:1. The question is how far down the chain we can go and still get good results for various release formats. It would be great to bust the compression, and therefore discover compression limits. Every codec has them. We know Redcode is great, but we also need to know how this fine variety of compression ratios works. Should not be a private discussion at all. I'm sure Red would benefit by people knowing the best ways to use the camera.

 

[James Brundige]

Hardly scientific, but Jim Hare shot some comparative shots that we were able to pixel peep at our Redusers meeting last week. To cut the long story short, if you are going to HD delivery, RED RAW 15:1 at 5k is as clean as it gets. You can't see any noise. You can see the noise at full 5k right until about 6:1 and it becomes academic for lower compression ratios.

The compression noise is most evident in shadows. A difficult scene with lots of shadows and detail (cat's fur) was chosen. ISO 800.

If you shoot outdoors or good lighting...you can get away with much higher compression before you can see compression artefacts.

Thanks for sharing this, Pawel, very interesting. I'm surprised 15:1 held up that well - good news. I'm wondering how 4K and 3K perform as well. I have need to use those for DOF issues. Did Jim Hare present any 4K stuff?

 

[Zeb B]

To the second level technical forum concept:
It is not me who has concerns about what is shown to who where. The concept was just an idea to allow legit Owners/Users to share raw experiences without the concern of how those experiences are interpreted by those who are not direct stake holders or competitors. I read on REDuser all the time that "this is RED's house - not the Internet". The Internet is raw and very unconcerned with getting banned or shit listed. REDuser is very self-aware. That is the reality friends.

If it's a non-starter idea then disregard. Love how quickly posts get personal here: "Charmer", VIP, Burning Man". Take a bow and let's get back to the technical conversation of Compression.

 

[Häakon]

Hi all,

Sorry for the delay in responding to this topic; I got booked for a job literally 5 minutes after I said I would set up a test shot for this, and between that and the holidays it's been pretty busy around here. That being said, I did want to shoot SOMETHING for this group as I promised I would.

I actually got a lot of feedback about this and I went in knowing fully well that no matter what I shot, it wouldn't be applicable/preferable/usable for every individual. That would be the case regardless, so I'm not losing sleep over it. Take this footage for what it is; hopefully you get something from it. If not, just move along. This is merely one scene with one set of parameters in one set of shooting conditions, and every setup is going to effect how compression will play in your final output.

With that in mind, I came away fairly surprised at the results.

Before we get to the footage itself, I'll throw out some of the requests I got for this test. The scene ended up being entirely different than what I had originally planned to shoot (I just made use of what I could on set), but I tried to include as many of the requests in one shot as possible. They included:

A low-light, exterior location
Rain
An open flame
Smoke
Skin and faces
Moving hair

I thought for a moment about how to pull all of these together on the fly, and the result is what you see. In hindsight, it probably would've been best to do this in front of a green screen, but time did not permit. Perhaps when I return home from the holidays I can do another setup, but that won't be another week at the very least.

I think a more detailed background with deeper focus would've been more appropriate for this test, but I was limited as to where I could shoot and tried to counter that with the rain. Unfortunately the rain didn't show up as pronounced as I had hoped (this is why we do tests, right? Guess I needed a test for the test!) as I think that is what shows off the compression changes the most. That being said, you can definitely still see the differences there. You can also see some interesting changes in the parts of the subject's hair that are intertwined and backlit. I did shoot this scene at every compression level available, but the resulting file size was far too large to upload anywhere, so I only included every-other-step in the final output.

Test was shot at 5K FF / 24fps at ISO 800. Absolutely nothing was changed in between takes except for adjusting the compression ratio. Footage was brought into REDCINE-X Pro for a very mild grade and exported to Prores 4:2:2 HQ for sharing (as requested). I am happy to make R3Ds available for those who want them, but I have no server space available so you'll have to provide a means to get it. Total footage shot was ~20GB.

In my personal opinion, the footage holds up extremely well until around 12:1, but it's not like it starts to get disastrous. Even at the "worst" level of 18:1 it is still plenty usable for a web presentation or the like. That's pretty significant for ultra-low-budget projects where disk space might be a legitimate concern. We've come a crazy long way from those DVCPRO HD days. Where it's going to hurt the most is in VFX applications (which is why I wish I had shot this in front of a greenscreen). I would like to do more tests now, because this leads me to feeling like I can relax my normal shooting preferences and be comfortable shooting around a 10:1 ratio in many circumstances. That's pretty major - a 2x savings over the 5:1 "standard." Surely one cannot make any sort of sweeping judgements based on a single isolated test, but I do find the results to be pretty impressive. My hope is that this may encourage others to do something similar so that we can begin to see how it plays out in a myriad of situations and build a mental database of what works best where. Of course if money, time, and space are no object I would shoot everything at the best quality anyway - but we know that is often not the case. Even Spider-man and the Hobbit aren't shooting in 3:1. Having the knowledge of where this particular compression affects the image most and how we can best exploit that is the idea. Hopefully you can take something beneficial from this experiment.

Here is a link to the footage... I don't know if there is a limit to the number of people who can download it, etc., but it would be great if someone could set up a mirror as I don't have the means to serve a large number of people:

http://www.megaupload.com/?d=STMI3KDU

I would like to take a moment and just say a huge thank you to RED for making a camera that is virtually noise free at this level; it really is such a pleasure to shoot with every time I pull it from the case. I can't state that enough.

Thanks for entertaining my attempt and please keep the discussion positive - everyone have a safe and wonderful holiday weekend if you are celebrating!

 

[Zeb B]

You are . . . THE MAN!

This very helpful and certainly encourages me to explore the subject much further. Thank you so much for you time and let's keep the dialog going!

Happy Day Off tomorrow!

 

[Frank Cueto]

Megaupload is requesting that I become a paid subscriber to download the file. can anyone post this somewhere else?

thanks.

-F

 

[Häakon]

Hi Frank,

Sorry... that wasn't the intent; it didn't tell me there were any limits when I uploaded it. The file is just under 2GB, so I'm not sure if there are other places that will provide such a large file for free. If so, please let me know. It did take all day to upload it on my current connection, though, so it will probably be a bit before I'm able to get it on something else.

Best,

Häakon

 

[Gerhard Riesenhuber]

Hi Häakon,

Thank you very much for sharing the testresults!!
Maybe you can split up the footage into 2 pieces under 1GB.

Best regards,
Gerhard