Noise and green screen / VFX

[Christoffer Glans]

First off, this is not a thread for complaining about noise, there's been enough of those.

I did a music video recently on my MX with green screen and the compositing artist complained about there being a lot of noise making it hard to key it cleanly. I though this was odd because I shot it at ISO320 and with enough light both on the green screen and the artist. The footage was downscaled to UHD 16bit TIFF sequences through DaVinci Resolve before keying and I used full premium debayer as well. Still, looking closely at the footage, there's a bit of texture that's hard to get rid of without advanced noise reduction.

Now I have upgraded to Dragon and we all know that it has a bit more noise. How do I shoot a green screen scene correctly to be able to get a clean key? What's the best settings etc.? I know that 35mm has been able to shoot with green/blue screens for decades, so there has to be a good strategy to get a clean key even with a lot of noise.

Thoughts?

 

[Hans von Sonntag]

I shoot often GS and get always good results. Besides the known techniques such as space lights for fill and lighting the green BG, meaningful distance to the screen, keeping green-spill under control with light and shade, etc... this are vital points to consider.

First off, expose fat, use ISO 250/320

Second, don't grade the footage, use Redlog Film and Dragoncolour, nothing else.

Third, don't apply any noise reduction. Let, if necessary, the flame/VFX artist do it as she/he needs it. They know how to tackle noise.

Fourth, ordinary 10 bit DPX will do the trick in most cases, even ProRes 444 is not as bad as one might think.

Fifth, use meaningful software, eg. Flame/Smoke, Nuke, Fusion.

Sixth, don't touch the original footage, match the contrast of the BG footage to the RLF footage. Only grade the final compositing to your linking.

Seventh, did I say use RLF and don't grade the footage?

Have seen students and people on a budget working there asses off trying to get a good comp done in AE. AE is a very good moving graphics tool. Besides pulling the matte, colour matching and grain management are the key to a good GS compositing. Use dedicated rack focus nodes for depth of field. In the AD boxes pixel spread is the wand of wands.



Hans

 

[Norbert Niederacher]

Time for
[h=3]REDucation![/h]
I believe, it would be great if RED offers One-Day-Workshops about this. I have heard a lot of complaints with Red Cameras/Greenscreen. Not every Shooter/Cameraman is really so well trained...

Thanks for the Info, Hans.

 

[PatrickFaith]

What I am doing now is clicking on "1" and "2" on the gioscope, and area light just enough to make sure i don't see any of the 1 & 2 shadings on the screen.

https://www.youtube.com/watch?v=tpl9G82hE0I

 

[Christoffer Glans]

I shoot often GS and get always good results. Besides the known techniques such as space lights for fill and lighting the green BG, meaningful distance to the screen, keeping green-spill under control with light and shade, etc... this are vital points to consider.

First off, expose fat, use ISO 250/320

Second, don't grade the footage, use Redlog Film and Dragoncolour, nothing else.

Third, don't apply any noise reduction. Let, if necessary, the flame/VFX artist do it as she/he needs it. They know how to tackle noise.

Fourth, ordinary 10 bit DPX will do the trick in most cases, even ProRes 444 is not as bad as one might think.

Fifth, use meaningful software, eg. Flame/Smoke, Nuke, Fusion.

Sixth, don't touch the original footage, match the contrast of the BG footage to the RLF footage. Only grade the final compositing to your linking.

Seventh, did I say use RLF and don't grade the footage?

Have seen students and people on a budget working there asses off trying to get a good comp done in AE. AE is a very good moving graphics tool. Besides pulling the matte, colour matching and grain management are the key to a good GS compositing. Use dedicated rack focus nodes for depth of field. In the AD boxes pixel spread is the wand of wands.



Hans

Yes, but as you must have read, I shot it correctly. I got an evenly lit green screen with no spill on the artists (they had black/red clothes and black hair with no reflective surfaces), the settings in-camera were ISO320, correct exposure within the histogram range.

In resolve, I did not choose REDlog, but instead I chose redgamma4, because first off, it was in histogram range without any clipped blacks or whites and second it went through the downscale/debayer process using all those colors balanced instead of applying them after the render before keying.

This was not going to be a green screen illusion imagery, but a motion graphics image with them as silhouettes, so there weren't any second applied contrasts needed based on backgrounds because such filmed backgrounds weren't used. The thing is that this isn't about how to create the illusion in post, but how to get a clean key out of Dragon material. I could blame the compositing artist not having enough skill to key the material and he is only using AE for this work, but having a 5K -> UHD full debayered TIFF 16bit sequence of the material should be quite enough for a clean key, but it still had noise in it.
If this is the case with MX, it will sure be bigger pain with Dragon and it's noise.

The only thing that created problems for him was the fact that the noise was in the way of a clean key. Is it Resolve that does something bad to the material? What is it? I mean if I put the material through a pass of neatvideo it should work better right?

 

[Matt Ryan]

I can tell you I shot greenscreen for a major rock band on Dragon, at 800ISO Low Light OLPF 6KHD, 3200k, 4:1 compression and had glass clean footage. The vfx guy said it was the cleanest, most detailed footage he had seen and that it was basically a one click key. So more importantly, what was your compression ratio and what was your color temp? Compression ratio on green screen makes a HUGE difference. Do not skimp here.

I hate to say it but Dragon IS way cleaner than MX. That is if you use the LLO olpf.... My ISO 500MX footage looks noisy compared to 800ISO Dragon with the LLO olpf and same scene...

 

[Peter Lyons Collister ASC]

When we shot Amaing Spiderman on Epic MX betas we shot more than 700 blue screen shots.... Sony Imageworks found the image clean. On Dragon I have probably shot another 600 shots on 6 more projects on GS and have not had a comp issue....

Could it be that the post people are processing the r3d incorrectly?

 

[Randall Weatherington]

Also, whenever possible shooting any color screen, separate screen lighting from foreground lighting.
550 nm green lighting on green screen only. 450 nm blue lighting on blue screen only. No spill onto the foreground.
Appropriate lighting on foreground subject. Important! Try not to let the white light on the foreground spill onto the screen.
White light contaminated screens are much harder to separate.
Obviously not all subject matter can be isolated from the screen as some foreground subjects may have to stand on the screen among other things.
The bottom line is to keep the screen as pure as possible without contaminating it with any other light.

 

[Gavin Greenwalt]

You didn't mention the most important variable with green screen footage: compression. If you overcompress R3Ds they'll be very difficult to key well. REDCode mostly compresses color not luminance. So the result is your footage "looks fine" at say 8:1 but then the keyer tries to operate on the chroma channels to perform a chroma key and the key is all flickery. For RED specifically: Use low compression, Use Full Debayer, half debayer results in weird color artifacts and rate the sensor for 320ISO. Daylight colored lighting works much better since the blue channel will be a stop less noisy.

Otherwise all of the regular rules of keying apply to Dragon as any other camera. 1) Light the foreground "normally". DO NOT throw a red backlight on the subject. Ugh... this happens so often. "There is green spill so I'm going to add a red backlight to cancel it out." No. I have despill correction to correct spill. Now I just have a red light I have to remove as well. 2) Keep the green as small as possible. If your actor never walks in front of something, don't put green on it, you'll just be adding to the green spill. 3) Shoot clean plates when you have time. 4) Don't denoise, the compositor has denoise algorithms that are as good or better than yours and they'll be able to decide when/if they need it. 5) Increasing the distance of your actors from your greenscreen doesn't actually in of itself help anything. If you need a 10 square foot green screen 2 feet behind them it'll be the same angular portion of the actor as a 100 square foot green screen 20 feet behind them. It does help about 20% of the time but it's not a "law" of green screens. People think it causes less spill because it's farther away... but you also have to pump exponentially more light onto a much larger screen which just cancels out the distance falloff. 6) Shoot with good lenses. Trying to key with chromatic aberration is a huge PITA. I seriously believe renting a great set of cookes with minimal color fringing will pay for itself in the keying. 7) Most "perfect keys" are really 90% rotoscoped so you're wasting your time anyway.

 

[Christoffer Glans]

Could it be that the post people are processing the r3d incorrectly?

I did the export for comp. It was a UHD TIFF 16bit sequence export with debayering at full premium. Redcolor4/Redgamma4 (within histogram range). I did the export through DaVinci Resolve, could it be that the software didn't export it clean anyway? Forced it to export at full debayering.

You didn't mention the most important variable with green screen footage: compression. If you overcompress R3Ds they'll be very difficult to key well. REDCode mostly compresses color not luminance. So the result is your footage "looks fine" at say 8:1 but then the keyer tries to operate on the chroma channels to perform a chroma key and the key is all flickery. For RED specifically: Use low compression, Use Full Debayer, half debayer results in weird color artifacts and rate the sensor for 320ISO. Daylight colored lighting works much better since the blue channel will be a stop less noisy.

We shot it at 4:1 compression. We lit with 5600K and did not use any colored backlight.

Here's a R3D trim of one shot. I have a hard time seeing the noise my comp artist complained about and I'm beginning to think that he isn't as good as he appeared to be :tongue_smilie:
Download

 

[Hans von Sonntag]

Christoffer, you know your stuff.

The weak part in your chain seems to be the VFX guy. Pulling a good key off Red originated footage is not rocket science if you use the cookbook that Gavin wrote down. I'm quoting myself, but AE is not the right tool for GS based compositing. Any seasoned VFX artist knows that and uses Flame/Smoke, Nuke or Fusion. Please check the postproduction workflow used in Mr. Collister projects he mentioned above. This could be a good starting point for future projects.

---

In my world, this is TVC-land in Europe, we use Flame and Nuke for such tasks. And yes, oftentimes there is a lot of roto work involved in various techniques. As a part-time AD Box user I'm amazed how quick the modular keyer delivers excellent result. But you have to know your stuff. Tools and skill matter. You don't shoot GS with DSLRs. Same applies to software and VFX artists.


Hans

 

[Hans von Sonntag]

Here's a R3D trim of one shot. I have a hard time seeing the noise my comp artist complained about and I'm beginning to think that he isn't as good as he appeared to be :tongue_smilie:
Download

Looks good to me.

But next time use RedlogFilm. As Graeme pointed out many times, RLF is the gamma curve in the Red universe that translates sensor information to 10 bit as lossless as possible. Same applies to Alexa's Arri C-Log.

Hans

 

[Craig Parkes]

Given that it seems like you did everything right/others say the footage looks good, I would say there is one other aspect that can create the impression that footage is grainy - that it's a UHD key being pulled.

One thing that catches out a lot of people doing this sort of work, especially in After Effects, is a lack of perspective of noise when looking at a 100% crop. This is because at 100% they are often viewing your 4K image on a HD screen that's only about 72 PPI from about 2 feet away - it's not a real life scenario, and if they are doing smaller res previews these are also inaccurate because they aren't quality downresses, just pixel binned 1/2 or quarter res samples

Then they often still can't playback a significant amount of footage in realtime, so a short ram preview will exacerbate their perception if noise because even the most minor static noise, when played in a short duration loop, will draw attention to itself.

So effectively what might seem noisey, or even a noisey key, is actually great footage but they are inspecting it much more closely, for a lot less duration, then they normally do with HD footage.

 

[Christoffer Glans]

Christoffer, you know your stuff.

The weak part in your chain seems to be the VFX guy. Pulling a good key off Red originated footage is not rocket science if you use the cookbook that Gavin wrote down. I'm quoting myself, but AE is not the right tool for GS based compositing. Any seasoned VFX artist knows that and uses Flame/Smoke, Nuke or Fusion. Please check the postproduction workflow used in Mr. Collister projects he mentioned above. This could be a good starting point for future projects.

Thanks :beer: but yeah, I think he is the weakest link here. Even though I know how to shoot it I did actually have another comp-artist who happened to be in the studio at the time look at the images and he said everything was good to go. Weird that my guy couldn't pull a clean key from this.

Out of Flame, Smoke, Nuke or Fusion, which would you say is best for keying footage?

 

[Matt Ryan]

I'm not a vfx guy and this is just dropping on keylight in AE but I don't get what issues the vfx guy was having. This literally took me 15 seconds and not an issue with the key and I don't see any noise. Needless to say your footage is fine and someone who actually does VFX should get a superior result to what I did. The only thing I can think of is maybe his monitor isn't calibrated and it's artificially bright, thus making it appear to have noise?

 

[Christoffer Glans]

Hahahahahahhahaahhaahhahahaah :lol:
That is so far from how the video is going to look, but awesome :beer:

Yeah, that's what I thought. I also think it's weird because as I mentioned, the footage isn't going to look like they are standing somewhere, it's silhouettes with details in them, so the need for a superb clean key wasn't really needed. Here's a frame from how the final output will look. Quite different from your image hehe
I'm not totally happy how the images turned out, he spent too much time on very basic stuff so I'm a bit disappointed, really. But we didn't have a large budget so I can't really complain, other than do it myself next time.

 

[Matt Ryan]

Hahahahahahhahaahhaahhahahaah :lol:
That is so far from how the video is going to look, but awesome :beer:

Yeah, that's what I thought. I also think it's weird because as I mentioned, the footage isn't going to look like they are standing somewhere, it's silhouettes with details in them, so the need for a superb clean key wasn't really needed. Here's a frame from how the final output will look. Quite different from your image hehe
I'm not totally happy how the images turned out, he spent too much time on very basic stuff so I'm a bit disappointed, really. But we didn't have a large budget so I can't really complain, other than do it myself next time.

hahaha that's funny. I keyed it and then was like, I need something for the background image and went with it haha. But the point is there's no issue with your footage and no reason a key wasn't super easy. Make sure to post the finished project once it gets released for the public, definitely looks interesting.

 

[Hans von Sonntag]

Out of Flame, Smoke, Nuke or Fusion, which would you say is best for keying footage?

I use Flame/Smoke. Others use Nuke and Fusion which I only know by sight. It depends what sort of job it is. Shot based compositing in feature land is mainly done with Nuke and to a lesser degree with Fusion. In commercial land compositing is often seen as part of client attended finishing, hence Flame and of course Smoke/Flame Assist (for the less demanding jobs).

All those 3 packages are designed to do compositing jobs with GS involved particularly well.

Hans

 

[Christoffer Glans]

Make sure to post the finished project once it gets released for the public, definitely looks interesting.

Will do :smile:

I use Flame/Smoke. Others use Nuke and Fusion which I only know by sight. It depends what sort of job it is. Shot based compositing in feature land is mainly done with Nuke and to a lesser degree with Fusion. In commercial land compositing is often seen as part of client attended finishing, hence Flame and of course Smoke/Flame Assist (for the less demanding jobs).

All those 3 packages are designed to do compositing jobs with GS involved particularly well.

Hans

I'm a bit of a swiss knife in that I do a lot of things other than directing, but that also means I cannot focus all my energy on a lot of things... can you name one of those that might be the best general purpose compositing software? If I want to learn one I need to choose one to put all that energy into. I'm moderate in AE so I'm not new to compositing, but I haven't worked with node based workflows except in Resolve.

 

[Gavin Greenwalt]

Could you post a short clip mot just a still? I wouldn't throw the vfx guy under the bus without seeing it in motion. I've shot greenscreen before and it looked great as a still but was really chattery in motion.