How to green screen in a forest

[Linda Barzini]

This BTS video of the Hobbit shows a florescent green screen in the background, yet the foreground has all kinds of greens as well.

Is this easy enough key to pull because the florescent green bg and more less saturated darker hued forest leaves are far enough apart?

I notice all the VFX on Guardians of the Galaxy 2 was shot on a blue screen.

So if you want to composite the background with forest/green leaves as foreground elements, you could use the bright green or blue backgrounds?

 

[Bob Gundu]

Most are switching to blue screens when compositing night scenes because the blue spill integrates better.

 

[John Brennick]

This aspect was one of the hardest parts about this show. On hobbit 2 we had this problem as well and luckily in these situations we were comping more trees in behind the trees so you if lost a bit of the detail you could get away with a more forgiving transparent leaf. But what would help in separating the greenscreen from the trees if it would work was the IBK. That was the main keyer most of use used and we would build a series of clean plates and track them in. However in sections that were really bad we had a combination of rotoing the fg and enviro would just replace those trees.

 

[paulcurtis]

There seems to be quite a disconnect at times between what the guys doing the keying need and what's shot on location.

An awful lot of stuff just isn't keyed but roto'd, especially with any 3D post work being done.

If you have a director that knows what they're doing then they'd shoot against white, black or whatever is needed for that shot. I believe Gareth Edwards (from a vfx background) is very capable of choosing what's right in a given situation. Then Roland Emmerich shoots people in front of physical set pieces and then screens behind - meaning that the keying work is limited to hard surface objects rather than hair and people - genius move.

I've keyed some horrendous screens. And Roto'd many others.

One issue with high luminance screens if that edges blend to a luma level that isn't the background, makes it very difficult to despill edges properly. I see in breakdowns all the time when the edges have been eroded in, probably because of mismatched luma levels.

My favourites are people in green suits and booms covered in green. Like you're honestly going to pull a key of a heavily shadowed cylinder or organic shape?

cheers
Paul

 

[Linda Barzini]

Great answers. Thanks.

It seems one of the problems with VFX is that some people are so talented they can do anything, but it takes a tremendous amount of work. VFX is underappreciated. I remember Rhythm N' Hues, the firm that did the VFX for Life of Pi, went out of business after that movie was released.

Even with the modern improved tools, I can't imagine having to roto entire scenes. Or dealing with the different luma values with transparent foreground elements. The Emmerich solution is genius. I've only played around with it a little, but have found blondes particularly difficult to key without shrinking the mask too much because its not so much spill but transparency as the problem.

I have noticed many heavy VFX films are filmed with RED cameras.

 

[Mark Spraker]

Isn't keeping this in check the responsibility of the visual effects supervisor? I know to people with no experience might not understand why, but fixing problems on set is a lot easier in most cases than dealing with the problems in post. Yes, it's more physical labor in a lot of cases, but if post production is easy, you can save time, money, and resources to dedicate to more complicated shots.

Rotoscoping can be absolutely mind numbing at times but has gotten easier with new software. I've heard of people spending months rotoing hair for a scene (I would die). Still, the VFX industry as a whole feels like an incredibly underappreciated position, especially at the higher levels where money can be thrown at a problem in post until it's fixed when it would have been easier for everyone involved to fix it on set.

And then there's this old gem: The client wants me to roto smoke?

 

[Matt W.]

Isn't keeping this in check the responsibility of the visual effects supervisor? I know to people with no experience might not understand why, but fixing problems on set is a lot easier in most cases than dealing with the problems in post. Yes, it's more physical labor in a lot of cases, but if post production is easy, you can save time, money, and resources to dedicate to more complicated shots.

Rotoscoping can be absolutely mind numbing at times but has gotten easier with new software. I've heard of people spending months rotoing hair for a scene (I would die). Still, the VFX industry as a whole feels like an incredibly underappreciated position, especially at the higher levels where money can be thrown at a problem in post until it's fixed when it would have been easier for everyone involved to fix it on set.

And then there's this old gem: The client wants me to roto smoke?

Granted I work mostly on the low end of the industry, but most productions don't seem to have supervisors on set and supervisors don't always know what they're doing either. There's an incredible amount of apathy in terms of how things are shot. Even on many of the highest rated shows on tv, art department doesn't know how to fill a laptop with a green or blue image, and they'll just use a green or blue desktop picture with all the icons and task bars still visible, or often much worse (taping the edges of the screen with green, dimming the screen only partially, using this weird tracking marker things, lots of comping over live screens, too, or you get a shot with some blue screens some white screens and some live screens all mixed together even inconstantly over the course of a scene which is which), and there's no care shown for what can be tracked and what can't. People just wave phones around all over the place with tons of occlusion, often you have one corner in the frame at most at a time and have to combine a position track with hand-animated perspective. (Usually a black screen is preferable, anyway.) A lot of notes come from the legal department, too, since no one checks to clear things anymore and a lot is shot on location these days. Sometimes someone on set might choose to shoot against a mural that looks cool and then it turns out it's not cleared, so you have to paint it out. That stuff happens constantly. Tons of beauty work, cosmetic fixes, split screens, time remapping, etc. Editors assume anything can be fixed in comp, usually it can, too.

Thing is, a day on a low budget set (indie feature, tv show, ad) costs about $250,000. That's over $20,000 an hour. I don't think that even includes actors' rates. So it's a lot cheaper to hire an artist for a day than a cast and crew for a minute or two to fix something. So saving time on set is not worth it. It's actually a money-saving tactic to not clean up camera marks or just leave the boom mics or stunt pads and wires and everything conspicuously in frame and then paint them out in post. Overall it saves money. Producers also want control in post so they'd rather not do things practically. This is true even in Marvel movies (I know one of the Marvel directors who started in tv and we discussed it).

It's less to do with appreciation or lack of appreciation than a financial concern and an issue of control. So it's not a bad thing, just a matter of how budgets break down.

 

[paulcurtis]

Like any profession in this world - there is a huge range of levels of talent within it. And age doesn't always mean better!

Also sets can be very expensive places and *only* the post guys see what happens in post - out of sight, out of mind. So it is a careful juggling act. Then each post house will have their own approaches and software preferences. But yes, most VFX artists are totally under appreciated. I think it's one of those industries that until you have to do it you don't realise how much troubleshooting and just damn clever solutions are cooked up all the time.

So the theory of vfx supervisor on set is solid. The reality can be different. And sometimes it *is* cheaper to fix it in post.

cheers
Paul

 

[John Brennick]

Isn't keeping this in check the responsibility of the visual effects supervisor? I know to people with no experience might not understand why, but fixing problems on set is a lot easier in most cases than dealing with the problems in post. Yes, it's more physical labor in a lot of cases, but if post production is easy, you can save time, money, and resources to dedicate to more complicated shots.

Rotoscoping can be absolutely mind numbing at times but has gotten easier with new software. I've heard of people spending months rotoing hair for a scene (I would die). Still, the VFX industry as a whole feels like an incredibly underappreciated position, especially at the higher levels where money can be thrown at a problem in post until it's fixed when it would have been easier for everyone involved to fix it on set.

And then there's this old gem: The client wants me to roto smoke?

Well with The Hobbit it was a really interesting situation. I agree in whole that the VFX Supervisor should always step and say "you know you're going to A... pay out the ass to fix this in post B.... probably get a better result if you just fixed it now and 4.... Probably push some vfx artist to want to blow his brains out trying to fix this right?" However more often then not I see producers or directors step in and say there's no time that they have to make the day... It's always viewed as if VFX has the ability to fix it then they will bare the burden. I question though if it's financially worth it. While yes the production crew can get really expensive... One thing thats often overlooked is the concept of opening a can of worms. Once you start to fix it in post, there's always going to be a note to make it more realistic. "The animation is different, the edges arent very good, the shader needs more work, the matchmove is slipping" these are all things that can drive the costs WAY up. Because these things usually end up going into overtime hours to get them done. And they can take weeks to do. That being said with The Hobbit, because Peter Jackson was filming in almost a run and gun style on his set it made it next to impossible to account for every angle and account for the green screen in every shot because it would interfere in other hero angles. He had so many Reds on set that he would place them all over the scene and shoot the scene all the way through and then just cut the film how he saw fit in the end. There were shots that came in where roto just wasnt a solution and they had to take it a step farther and matchmove some of the actors head and create a CG wig. In those shots the camera was looking straight up into the ceiling of the warehouse and the spacelights were blasting through the actors hair. The hair was so clipped from the lights and flares going over their heads that the only way to do it correctly was to matchmove the head and render a CG wig. Now add 48fps and stereo to all of this.... And it's a recipe for a bad day, everyday. We were on 100+ weeks at the end and more often then not a good portion of those hours were fixing stereo issues from the key. None the less it got done and some people go there specifically for that kind of work and go there for that kind of overtime.

 

[Neil Abeynayake]

Great answers. Thanks.

It seems one of the problems with VFX is that some people are so talented they can do anything, but it takes a tremendous amount of work. VFX is underappreciated. I remember Rhythm N' Hues, the firm that did the VFX for Life of Pi, went out of business after that movie was released.

Even with the modern improved tools, I can't imagine having to roto entire scenes. Or dealing with the different luma values with transparent foreground elements. The Emmerich solution is genius. I've only played around with it a little, but have found blondes particularly difficult to key without shrinking the mask too much because its not so much spill but transparency as the problem.

I have noticed many heavy VFX films are filmed with RED cameras.

Rhythm N' Hues went out of business because the "Venture Capitalists" who invested in that company did not understand anything about 3D compositing and finishing & wanted to squeeze every ounce of energy or "money" they could out of that company. They fired almost all creative talent including key animators and principles out of that company once the project was completed and what or who were left could NOT create jack shit when needed.

I had friends who worked for that outfit. They were all gone in the first salvo of lay-offs.

 

[Mike Nagel]

I have noticed many heavy VFX films are filmed with RED cameras.

one would assume that given the extra resolution, but even way more are shot on Arri.

Ridley shot latest Alien Covenant on standard Alexa (not even Alexa65), and u would think (given the massive amount of VFX) that this movie would absolutely qualify for RED, considering Ridley has RED experience...

Re green screen: the key (pun intended) is to shoot the screen correctly, in order to make keys much easier. IMO, avoid keying hair at all costs and or humans in general, unless it's a long shot, then the (problematic) edges won't be noticed at all (because of the distance)...

There's many types of green screen surfaces... I have the one that is very saturated (similar to the one in the pic above)... many swear on it, others say it actually introduces more problems...

... and the real tip: wait until Lytro is a reality... that is an absolute game changer. (but the data rates are insanity)

 

[Mike Nagel]

if it would work was the IBK. That was the main keyer most of use used and we would build a series of clean plates and track them in.

IBK as main keyer for trees on green screen or in general on that show ?

Did you use that in combo with Keylight and/or Primatte or other keying methods ?

 

[Phil Holland]

Great answers. Thanks.

It seems one of the problems with VFX is that some people are so talented they can do anything, but it takes a tremendous amount of work. VFX is underappreciated. I remember Rhythm N' Hues, the firm that did the VFX for Life of Pi, went out of business after that movie was released.

Even with the modern improved tools, I can't imagine having to roto entire scenes. Or dealing with the different luma values with transparent foreground elements. The Emmerich solution is genius. I've only played around with it a little, but have found blondes particularly difficult to key without shrinking the mask too much because its not so much spill but transparency as the problem.

I have noticed many heavy VFX films are filmed with RED cameras.

A good VFX Supervisor and decent relationship with the Cinematographer go a long, long way in lighting and pulling great keys. Sometimes logistics get in the way. Sometimes the problem is a beast to deal with. Sometimes you need to go with what "works best" even though it doesn't always look ideal.

RED works well for VFX for a lot of reasons. Some of it is resolution, some comes in the form of general pixel quality and color depth. Especially with the newer sensors.

Side note. I served for 11.5 years at R&H (left about about 1.5 years before the bankruptcy, but did work on Life of Pi). It was a rather complicated series of events that led to what happened, but it truly didn't have anything to do with that specific film.

John's moved onto some newer things which I've been fortunate to shoot on occasionally. Looking forward to his newest VFX adventure, PHD.

 

[John Brennick]

IBK as main keyer for trees on green screen or in general on that show ?

Did you use that in combo with Keylight and/or Primatte or other keying methods ?

On that show we were throwing the kitchen sink at it to get whatever was the best result. So while many of us were using the IBK there were quite a few who just had great results with keylight as well. But from my experience IBK was a life saver so a lot of us there it was the first keyer of choice. Because it's not just doing a key based on the color it's also doing that in combination with a difference matte. So one of the techniques we would do would be to basically paint our clean plates and reproject the clean plates back onto cards and use that as the Color input for the IBK. Specifically, I would use that mostly for the hair and keymix it in with separate keys. Primatte would generally always be the core matte. But there were a lot of tricks that were being passed around where people would figure something out that works great for one shot and the next day we're all trying it with our own to get those last little hairs. Because you're dealing with stereo any disparity between the keys between the two eyes would always get kicked back.

 

[Mark Spraker]

Wow, lots of excellent insight in this thread. Having never officially been on a 'real' movie set (outside of what I manage to stumble together on my own) I really appreciate the deeper look into this side of the industry. And it's nice to actually have more information on what exactly happened with Rhythm N' Hues...

 

[Linda Barzini]

FWIW, I think the Hobbit movies look amazing. I have been obsessed with watching them lately. It started with all the camera movement, then lighting, then compositing. It is nice that Peter Jackson put up a nice BTS series on youtube, so you can see the tricks. I was really surprised that they needed to add some much red to the actors so they would print as regular skin.

It would be great to see a BTS of the VFX team on that project, but I expect it would just be a litany of cursing.

 

[Marc Wielage]

Some very, very good information in this thread. I salute all of the hard-working VFX artists who put up with this crap on a daily basis. Your brothers in post share your pain.

 

[Mike Nagel]

On that show we were throwing the kitchen sink at it to get whatever was the best result. So while many of us were using the IBK there were quite a few who just had great results with keylight as well. But from my experience IBK was a life saver so a lot of us there it was the first keyer of choice. Because it's not just doing a key based on the color it's also doing that in combination with a difference matte. So one of the techniques we would do would be to basically paint our clean plates and reproject the clean plates back onto cards and use that as the Color input for the IBK. Specifically, I would use that mostly for the hair and keymix it in with separate keys. Primatte would generally always be the core matte. But there were a lot of tricks that were being passed around where people would figure something out that works great for one shot and the next day we're all trying it with our own to get those last little hairs. Because you're dealing with stereo any disparity between the keys between the two eyes would always get kicked back.

John, thanks for the insight.

Wow, totally forgot that they also released a 3D version... what a pain.

So, if you can comment, why would some of these shots on such a high end franchise (and considering PJ and his team have massive VFX experience) not be shot on a background that would be easier to key ? Is that an oversight, negligence or was this done on purpose as green edges / spill would blend somewhat by default with the foliage of the trees ?

FWIW, I personally thought that the Hobbits trilogy was a better franchise than the LOTR - not a better story, but better executed all around. Did not expect that when I heard initially that they're gonna stretch that tiny book over 9 hours and 3 movies ;-)

Although one thing I'll say as I cannot understand this on a franchise where everything else is of such high quality: there were a few select shots where the animation of the VFX werewolves (whatever the Orks are riding) was absolutely horrible. Like a C64 game. 3 frames animation. Same thing in LOTR when Gandalf and the crew ran through the Dwarf caves/mines in the 2.5D iso shot - but that was 10 years earlier. Never understand how something like that goes through QA/QC when everything else looks so slick...

 

[paulcurtis]

In those shots the camera was looking straight up into the ceiling of the warehouse and the spacelights were blasting through the actors hair. The hair was so clipped from the lights and flares going over their heads that the only way to do it correctly was to matchmove the head and render a CG wig.

+1 bazillion for these

Tracking bits of hair and painting over bits of background that shouldn't be there, *especially* lights through hair is a favourite.

It's an impossible situation - you can't stifle what a Director wants because it's a bit hard though. Some of the best directors (for those types of vfx movie) are the ones who have been through that side (David Fincher, Wes Ball etc,.) and have an innate understanding of what a shot would mean later on.

On large films there can be quite a hierarchy between Director and Artist as well. Lots of middlemen and women who are not always making things more efficient. I remember sitting in makeshift office back in soho days with a producer, because there was no other room. And it's amazing how efficient things become when you can chat directly and show examples, especially when the director comes in who is used to people previously just saying yep, no problem. I think most directors are pragmatic and if option A and option B is explained the choices are usually the right one. I also feel there is an incredible amount of unnecessary work carried out these days - from sky replacements to digital makeup that just isn't important to a story.

cheers
Paul

 

[PatrickFaith]

For forest and weeds, I prefer not to shoot with a greet screen but have a depth pass from the camera (i.e. the camera is moved sideways before it is fixed, then a point cloud solution is made from that and stored in a openexr 2.0 format or z-depth format). The only problem I've had with this in forest and meadow scenes is if there is too much wind which complicates the depth calculation. This turns out in post to be about 50x less manual work since you just specify the depth range in a mask, and magically everything not in that range is removed. Older but still valid example in nuke: https://www.youtube.com/watch?v=19w3vkFp5X0