OK - the RCX vs Color vs Redline/Clipfinder debayer noise-test when preparing images

[Gunleik Groven]

I have done a couple of tests around how to prepare a project for an online in Color.
I have a couple of projects to finnish, and there are all these new applications/plugins and options to consider when preparing an online.
There have been some discussions around grading directly from RAW in Color vs DPXs and RCX has been praised for all its promises.
Shortly put: I had to test this anyway before deciding what to do, and there's really no reason not to share my findings.

Methodology (or whatever one should call it...):
I have taken the same tungsten-lit frame and developed it with these settings:

Computer:
Octo-mac with ATI and SnowLeopard

Render-settings:
Camera RGB (in Clipfinder) or RedColor (in Color and RCX)
I've used the latest Color Beta plugin with the new color science (beta 4.0 v2)
No noisereduction or sharpening applied.

RedLog
ISO 320
Exposure -0,4
Kelvin 5600

All other settings flat
I rendered out dpx's with Color, Clipfinder and RedCineX

I chose redlog, as that is a very gradeable startingpoint, even though RedGamma might be better for some work. I'm just not yet too familiar with RedGamma, so I chose to stick to RedLog for familiarity. That shouldn't greatly affect the noise charracteristics.

In Clipfinder I turned sharpening off and used the Mitchell to scale to 2k

For all applications I chose the highest debayering algorithm available.

Then I had a look at the separate channels in Shake and in Color from the resulting DPXs.

I'll post the rendered DPXs for you guys to look into and some peeks into what's happening in the blue channel (for now) in the different applications.

I know I can make these frames more noiseless by tweaking the settings, but that wasn't really the point of this test.

I must admit that one of the results surprised me slightly.
And that one result was worse than I'd have guessed.


Edit:
I'll post the same results from RedRocket over the weekend.
AND some charts done wth the same tests next week.
I think a high detail mesh-like image will be particularily telling from what I've seen so far.

I did test to send M and F proxies to Color to see if there was any difference when color scaled down or up from the proxies.
Except making Color more sluggish, the F proxies didn't have any advantage. (As expected)

I will render out 4k frames in Color and compare to 4k frames from the others, but that's not for tonight...

Gunleik

 

[Gunleik Groven]

Ok. Here goes

DPX 1 - Color
http://www.byacserv.com/noise/color_01.dpx

Zoomed in on the blue channel from the Color-generated DPX - Crop:
http://www.byacserv.com/noise/peek/color_01_blue.png

DPX 1 - Clipfinder
http://www.byacserv.com/noise/cf_01.dpx

Zoomed in on the blue channel from the Clipfinder-generated DPX - Crop:
http://www.byacserv.com/noise/peek/clipfinder_01_blue.png

DPX-1 RedCineX
http://www.byacserv.com/noise/rcx_01.dpx

Zoomed in on the blue channel from the RedCineX-generated DPX - Crop:
http://www.byacserv.com/noise/peek/rcx_01_blue.png

DPX-2 - Color
http://www.byacserv.com/noise/color_02.dpx

Zoomed in on the blue channel from the Color-generated DPX - Crop:
http://www.byacserv.com/noise/peek/color_02_blue.png

DPX-2 - Clipfinder
http://www.byacserv.com/noise/cf_02.dpx

Zoomed in on the blue channel from the Clipfinder-generated DPX - Crop:
http://www.byacserv.com/noise/peek/clipfinder_02_blue.png

DPX-2 RedCineX
http://www.byacserv.com/noise/rcx_02.dpx

Zoomed in on the blue channel from the RedCineX-generated DPX - Crop:
http://www.byacserv.com/noise/peek/rcx_02_blue.png

The renders from Clipfinder should be the same with all the traditional renderers, like Monkey extract, redalert etc.
Didn't test this,but I take it more or less for granted.

Take down the dpx's for yourself and try to veryfy my findings.
... Or do similar tests yourself.

Cheers!

Gunleik

 

[Gunleik Groven]

If there's an interest for it, I'll post some 1-frame r3d's recorded with B30 alongside DPXs and cut outs over the next week.

 

[Ben Brainerd]

Nice to see someone who does their tests in a fairly un-biased way.

I find it interesting that there's so much gamma variation between the different programs, considering that they're all, in theory, outputting to the same gamma setting.

If nothing else, I think we might be able to finally put the "RedCine-X produces sharper images than Color" argument to bed, since for the life of me, I can't really tell the difference, even super-zoomed in.

Were these 4K frames originally? Or 2K?

 

[Gunleik Groven]

They were all 4k

And you've pinpointed my 2 surprises...

I'd have thought that RCX performed a lot better than Color, and I'd have expected RCX and Clipfinder/RA to come out with the same gamma.

My conclusions are though:
I'll continue to produce my dpx's before grading for the time being...
RCX conform is not the way to go as of yet.
But it will get there!
Color is still quite cool for grading DPXs
And RedLog is very gradeable for many deliveryscenarios...

I'd love to add Scratch, SMoke and Nucoda to the soup, BTW if anyone wants some test files and will post their resulting dpx's.

 

[David Battistella]

Thanks for doing this test.

It proves that Apple Color renders a very tight image (aside from the gamma issues as described by ben)

What was most interesting is that if you open them all in Photoshop and apply autolevels you will find that the frame with the least amount of Blue Channel noise is the CLIPFINDER image. The color and RC-X images are about identical.

That's interesting. You might have discovered that the Mitchell setting in Clipfinder renders less noticeable Blue channel noise.

I hope to god this thread stays clean.

You've mythbusted what ben and I have been talking about for a while. Apple Color is a very competitive solution...

David


David

 

[Ben Brainerd]

A quick note, in line with something David said, posted in a hurry to head off any potential nay-sayers: The Color image, if you just look at the blue channel in comparison to the blue of the RC-X image, looks like it's more noisy. It's not, really. If you level it out (I also just popped a Photoshop auto-levels on) Color and RC-X have an equivalent amount of noise.

That's all for now. I've got some more thoughts, but got stuff to do right now.

 

[Gunleik Groven]

From my previous experience, I would have expected Clipfinder to come in first, then RCX, then Color (at least as to the criteria I set for a technical pass, where sharpening is kinda unwanted - I prefer to add sharpening related to the delivery - be it SD, HD or 2k, and def don't want that in the initial pass, but YMMV on that subject...)

It does - however - confirm a couple of gut-feelings I've had on more informal tests I've done on footy lately.

RCX behaves more or less like Color - and does thus yet not hold up for a technical pass for a cinematic or high-end TV release IMHO

Color with RAW is also not yet an optimum solution, but holds up much better to RCX than I'd expected. As far as I can see - it's identical to the RCX results, if one takes the gamma-issue into account. To do a conform with RCX does thus have no advantage over going directly to Color - if this limited test should stick. (I'll do additional tests over the next two weeks. More on that later...)

Thus David, given our different criteria, I'd call it a draw between us on our discussions on the subject.

I am looking foreward to the RedRocket renders I'll se tomorrow, both through Clipfinder and from RocketCine-X (SHOULD be the same AFAIK...)

The question is: Are these results only due to controll over the scaling algorithm or is it more to it?

I'll make some 4k DPXs to check that.

I'll render out a 2k set, where I whiteballance prior to render.
I have a hunch as to what gives the cleanest input for Online, but we'll see.
Maybe that will differensiate the workflows more?

Finally I'll apply the same grade to a similarily set RedRoom and Clipfinder DPX to see how that effects noise and latitude.

Over the next two weeks I'll be doing a number of codec, workflow, sensor and exposure tests with M Red1 and also comparative tests with R1 vs 5d and 7d for a TV series. If I am allowed to publish the results, I'll do it. With single frame RAW images to go by the tests, for others to dig into the current image-workflow issues.

Sounds interresting?

I am temted to fire up old RC for the comparison, but that has more academic than practical value, I plan to leave it alone - if I'm not asked loudly enbough to put it in. These tests take some time to do after all.

I am particularily looking foreward to some gradient ramp cards and resolution cards under different lighting and exposure conditions.

M Most has stated that the 2 extra bits in RAW, can be good to have for latitude and precission, over 10-bit log DPXs. I am sorta surprised by that, as I think you lose a lot of what you might gain through added noise - in Color. I think I can dig a lot more noisefree images out of the low-lights, and still protect the highlights by doing a DPX pass first, but I will have to test it more formally now.

I'll try to think up ways to test for it.
Suggestions?

It would be interesting to re-visit Scratch, as Asimilate used to write their own debayering routine. But I am unsure if that's still the case with the rocket-accelerated Scratch...
Any takers on that?
Lucas?
I have particular ABC comparisons I would like to test on the current Scratch, if I was issued a demo lisence. Both with the Rocket and without.

BTW. It is not my intention to deem any workflow "right" or "wrong". What I dig about RED1 is that you can choose the appropriate workflow for the current production at hand.
What I want to look at are trade-offs and advantages of different workflows.

Rendering directly from cam-proxies in FCP is not a deliveryoption for the scenarious I'm diving into, so I'll leave that out.

Anyhow.
Thanks for the heads up so far.

Gunleik

 

[Mike Gross]

Clipfinder uses REDline to render anyway right?
Would that make it the same as redcinex?

From the images, the color looks different so it's hard to compare apples to apples.

 

[David Battistella]

His is great because your approach has scientific curiousity and genuine wanting to know over wanting to prove.

I have a series of R3D from both sensors on old and new science. I think it might be best to wait for the new QT codec, a new version of redline and the SDK so we could add Baselight, lustre and some others to this test.

The old redcine, as we know produced the nicest DPX files we could start with. I'm sure Scratch would be really good because they wrote Redcine so that would probably translate.

Great work.

I don't want to see a "winner" I just want to know what to expect from the tools.

David

 

[Gunleik Groven]

I don't want to see a "winner" I just want to know what to expect from the tools.

David

Thanks!

Me 2

Uploading the 2k/4k test @ 5600k and 2000k right now.

I must say, I got some surprises...

 

[Gunleik Groven]

The 4k and scaled 2k @ 5600kelvin and 2000 kelvin test

The 4k and scaled 2k @ 5600kelvin and 2000 kelvin test

First:
Please download and watch locally....

So here comes part 2 of the pixel-peeking debayer/render and codec test -

1st off, I wanted to check if I could use Color in 4k mode, to downsample the images to 2k/1080 after online.

I also wanted to have a closer look at what the new color science means when one develops at different kelvin settings, so all the following tests have been done @ 4k @ 5600k and 2000k. I chose 2000k over 3200k, deliberately to exggarate whatever happens with the K slider.

I have used a very low exposed image of Oslo. While the earlier posts were shot with build 20, these are shot with an earlier build. But as this is a test on how the codec and debayer & rendering processes work, I think these are still valid.

The results were IMHO very interesting.
Again the Color and RCX renders were almost identical, except from levels as far as I can see, so I guess I in the following tests just will skip the RCX.
There's interesting things happening in the blue and red channel too in these tests, but have a peek yourself on the DPXs, if that's of interest.
There are both (for me) expected and unexpected results here.
The most positive thing is how the NewColorScience seems to mature, compared to last time I did some pixel-peeking into it...

Settings (except for the Kelvin)

Redcolor (RCX/Color) or CamRGB (Clipfinder)
No DRX (That's where I got one of the interesting results from Color/RCX)
320 ASA
-0,4 exposure
Highest available debayer settings
No noisereduction or sharpening applied


Here goes the 4k @ 5600k bunch

Color 4k project DPX 5600k
http://www.byacserv.com/noise/4k_and_2k_5000k_and_2000k/color_4k_5600.dpx

The crop:
http://www.byacserv.com/noise/4k_and_2k_5000k_and_2000k/peek_4k_2k/color_blue_4k_5600.png


RCX 4k project DPX 5600k
http://www.byacserv.com/noise/4k_and_2k_5000k_and_2000k/rcx_4k_5600.dpx

The crop:
http://www.byacserv.com/noise/4k_and_2k_5000k_and_2000k/peek_4k_2k/rcx_blue_4k_5600.png


Clipfinder 4k project DPX 5600k
http://www.byacserv.com/noise/4k_and_2k_5000k_and_2000k/cf_4k_5600.dpx

The crop:
http://www.byacserv.com/noise/4k_and_2k_5000k_and_2000k/peek_4k_2k/cf_blue_4k_5600.png


And now: 4k @ 2000kelvin

Color 4k project DPX 2000k
http://www.byacserv.com/noise/4k_and_2k_5000k_and_2000k/color_4k_2000.dpx

The crop:
http://www.byacserv.com/noise/4k_and_2k_5000k_and_2000k/peek_4k_2k/color_blue_4k_2000.png


RCX 4k project DPX 2000k
http://www.byacserv.com/noise/4k_and_2k_5000k_and_2000k/rcx_4k_2000.dpx

The crop:
http://www.byacserv.com/noise/4k_and_2k_5000k_and_2000k/peek_4k_2k/rcx_blue_4k_2000.png

Clipfinder 4k project DPX 2000k
http://www.byacserv.com/noise/4k_and_2k_5000k_and_2000k/cf_4k_2000.dpx

The crop:
http://www.byacserv.com/noise/4k_and_2k_5000k_and_2000k/peek_4k_2k/cf_blue_4k_2000.png


And now for the scaled to 2k images


2k scaled @ 5600k

Color 2k project DPX 5600k
http://www.byacserv.com/noise/4k_and_2k_5000k_and_2000k/color_2k_5600.dpx

The crop:
http://www.byacserv.com/noise/4k_and_2k_5000k_and_2000k/peek_4k_2k/color_blue_2k_5600.png


RCX 2k project DPX 5600k
http://www.byacserv.com/noise/4k_and_2k_5000k_and_2000k/rcx_2k_5600.dpx

The crop:
http://www.byacserv.com/noise/4k_and_2k_5000k_and_2000k/peek_4k_2k/rcx_blue_2k_5600.png


Clipfinder 2k project DPX 5600k
http://www.byacserv.com/noise/4k_and_2k_5000k_and_2000k/cf_2k_5600.dpx

The crop:
http://www.byacserv.com/noise/4k_and_2k_5000k_and_2000k/peek_4k_2k/cf_blue_2k_5600.png


And finally: 2k scaled @ 2000 kelvin

Color 2k project DPX 2000k
http://www.byacserv.com/noise/4k_and_2k_5000k_and_2000k/color_2k_2000.dpx

The crop:
http://www.byacserv.com/noise/4k_and_2k_5000k_and_2000k/peek_4k_2k/color_blue_2k_2000.png

RCX 2k project DPX 2000k
http://www.byacserv.com/noise/4k_and_2k_5000k_and_2000k/rcx_2k_2000.dpx

The crop:
http://www.byacserv.com/noise/4k_and_2k_5000k_and_2000k/peek_4k_2k/rcx_blue_2k_2000.png


Clipfinder 2k project DPX 2000k
http://www.byacserv.com/noise/4k_and_2k_5000k_and_2000k/cf_2k_2000.dpx

The crop:
http://www.byacserv.com/noise/4k_and_2k_5000k_and_2000k/peek_4k_2k/cf_blue_2k_2000.png


I won't discuss the results as yet, but I can say I learned a lot from this.
The most interesting was maybe that it wasn't the scaling algorithm that made the biggest difference, even though I had started to suspect that.
What exactly is going on in the unexpected results, would be very nice to get a comment from Deanan/Graeme on.

But look for yourselves....
Still a couple of planned tests to go, though...

I am VERY curious as to how RedRocket holds up in these comparisons.
I'll check tomorrow!

 

[Sanjin Jukic]

Gunleik,

forget all those "old" tools...

RedCine-X is only way to go together with upcoming updates of those "old" tools with the newest SDK that can catch the latest RED workflow development.

Less noise and more pleasant fun in every direction with all your footage, old and new, shot with M or M-X...

 

[Gunleik Groven]

Gunleik,

forget all those "old" tools...

RedCine-X is only way to go together with upcoming updates of those "old" tools with the newest SDK that can catch the latest RED workflow development.

Less noise and more pleasant fun in every direction with all your footage, old and new, shot with M or M-X...

Except for the fact that as of today you cannot yet conform to RCX, I guess you haven't looked at the images...

 

[Sanjin Jukic]

I guess you haven't looked at the images...

Yes I did it.

 

[Gunleik Groven]

Then what we see differs.
But that wouldn't be a unique happening. -

I have great hopes in RCX, though!

 

[Gunleik Groven]

or...

RCX @ 4k 5600 kelvin, blue-channel-crop without noise-reduction:

Clipfinder @ same settings:

And scaled to 2k:

RCX - blue channel crop:

Clipfinder:

But that's maybe just me...

 

[Sanjin Jukic]

Gunleik,

right maybe that's just you.

Something is wrong with your processing methodology or software/hardware set up.

Have a look below there are pretty noise clean crops from 4K tiff that were snapshots from RCX.

It was shot handheld on R1-M in the car drive from back seat with Voigtleander Nokton 50mm f/1.1 @ wide open f/1.1 with extremely short DoF.

Crop from 4K original that shot handheld from the car drive on R1-M with Voigtleander Nokton 50mm f/1.1 @ wide open f/1.1 with extremely short DoF.

Detail from RCX 5600K,...

Crop from Shake 4K 5600K,...

Crop from 4K original that shot handheld from the car drive on R1-M with Voigtleander Nokton 50mm f/1.1 @ wide open f/1.1 with extremely short DoF.

Detail from RCX 3200K,...

Crop from Shake 4K 3200K,...

 

[Gunleik Groven]

Sanjin.

I can get noisefree images from all these apps, too. That's not the point.

The point is looking @ how differently they treat the images.

I have stated my setup and settings for all the tests and am fully open to critisism.

What I am posting is not grades where blacks are pushed down or other stuff done to make them look glossy, but basically looking at details and noise levels - particularily in the blue channel with different tools with similar debayer settings.

I am looking for differences, not beautifull images.
That's how I can make educated choices regarding workflow options for myself and others...

But maybe I SHOULD try 16 bit tiff exports. The reason I haven't is that that is not a viable option for the projects I'm currently involved in and investigating workflows for...

I hope you have seen that I have posted the DPX's as well. They actually don't look all bad. But they are different starting points for a grade. I am looking for these differences.

Cheers!

 

[Sanjin Jukic]

What I am posting is not grades where blacks are pushed down or other stuff done to make them look glossy, but basically looking at details and noise levels - particularily in the blue channel with different tools with similar debayer settings.

Cheers!

Gunleik,

I didn't make them glossy and my set up is from (default 5600K) with added 3200K option how it was showed at RCX app images without any further adjustment.

If you want to see them in original 4K tiff size let me know and I'll sent you a link to download.




Also have a look at the another example that is dropped as R3D QT H proxy in FCP and then sent to Color.

It's shot handheld with Leica Noctilux 50mm f/0.95 wide open @ f/0.95 (from October 2008).

Also no any adjustment in FCP or Color.

It is as it is from my desktop snapshots.

Crop from Color (Monitor 1, 8 bit Apple 30" Cinema Display), shot handheld with Leica Noctilux 50mm f/0.95 wide open @ f/0.95 (shot in October 2008).

Crop from Color (Monitor 2, 30 bit HP DreamColor), shot handheld with Leica Noctilux 50mm f/0.95 wide open @ f/0.95 (shot in October 2008).