Scarlet VFX video workflow and breakdown.

[chris morgan]

i posted our little VFX video in the 'shot on red' forum with a brief explanation, and since i've had a few people ask for more info on the workflow and how we did certain bits, i figured i'd write up a quick breakdown of how it all went down, man.

this is the video, a product video for a biotech company called spectrum labs here in LA:

i come from a 5DII background recently. done myriad product videos for Native Instruments, as well as music videos and other random stuff. i used to transcode everything from h.264 to ProRes422 for my edits, and worked in final cut studio 3 for the most part, having been an FCP user since it was available.

having said all that, this project worried us a little from a postproduction workflow perspective, as we were going into it having heard the RED postproduction horror story rumors. visualising the VFX stuff and knowing the amount of after effects, motion tracking and so on that would be required, it was a little intimidating.

we'd had CS5.5 for a while, and now after having used the CS integration, not to mention the asset management aspects, i'll never go back to FCP. but more on that later.

we had pretty much zero technical preplanning. we knew what we wanted to achieve, and were familiar enough with using matchmoving software and integration into a 3D postproduction environment that we figured we'd get by. this worked out for the most part, but we'll definitely be changing a few things on-set to make our future matchmoves and 3D integration a little more seamless. scaling ended up being the biggest hassle...

THE LAB

we shot the entire lab sequence in a garage in Bel Air. we built the set from scratch, assembling the main 'boxes' out of pressboard and 2X4. the lightbox around the lab is built from 1X8 boards, notched on the top to allow us to slot in frosted lucite. that's all lit from within using 5500K fluorescent bulbs. we ripped apart some old fixtures and used the zappy guts... bolted them to our enclosures and whammo. filled in a few dark spots with 5500K CFLs and some tinfoil.

the lab equipment was primarily supplied to us by the company, and is mostly defunct stuff, but it still lights up. between that and the copious amounts of glassware filled with varying amounts of food colouring and dry ice, the lab gear helped sell our set a lot more. beyond that, we got creative with LED light sets from Home Depot, as well as various fixtures mounted within the functional set pieces as well.

we also assembled a PVC lighting grid which is suspended from the garage ceiling on chains. this is only being used to hold Home Depot clamp-on aluminium par cans, which we gelled, black foiled, and used primarily to light up the various lab equipment and other features we needed to throw a little more light on. we also used one of those clamp-on par cans with a UV blacklight bulb that we flagged off and shone directly on to martin (our scientist) to give him a bit of a glow, ostensibly from the holographic projection system when perceived dramatically.

there are two American DJ VIZI-SPOT LED moving-head fixtures on either side of the lab under DMX control from a macbook pro using DMXIS software. this allowed us to dial in a GOBO pattern and light movement that evoked a sense of some sort of projection system responsible for the 3D screens. we also filled the room with a haze via an American DJ smoke machine, often waiting 10-15 minutes for the wispiness to diminish. we didn't want to compromise the contrast of the shot by having it be too hazy, and didn't want the haze itself to be visible, but there's a 'golden moment' when the haze gets to just the right consisency.

there's actually a pile of behind-the-scenes and lab-building photos on our facebook page if anyone is remotely curious about how we put everything together. you can check it out here - just click on our 'photos' and check out the 'our first set construction' and 'behind the scenes of dr. khameleon' galleries.

SHOOTING

everything was of course shot on our new best friend, Scarlet #01650, "nostromo". we did a black shade every day before shooting (although the final video is comprised of one day's shots for the most part.) primary lighting was done using two FloLight 1'X1' LED daylight panels. these are super-controllable and dimmable and worked really well in the tight shooting environment. the build quality of the FloLights leaves a lot to be desired, but functionally they've been great.

camera movement wise, we used an 8' Kessler Crane which is the most rock-solid nuclear-war-surviving bit of kit we own. the k-pod is a beast. but our most critical cheap element ended up being the dolly. we were using a rigwheels-based dolly on PVC pipe. for most dolly-style shots this has worked fairly well. it's never butter smooth, but for the most part we can cut around it. we had the usual bumps, wiggles, and jitters that we always have with this dolly, but as you'll soon read this ended up being something that nearly caused me to cut off my own head once we made it to post.

we boomed the whole thing with a Rode NTG-3 going through a Native Instruments KA-6 for phantom power, and then directly into the scarlet. we had ordered a startling amount of egg crate to cover the garage with to help with reverberation and sound, but it didn't make it in time, so the audio leaves a bit to be desired.

the newscaster was shot in two different locations. the opening was shot in a small home theater, the closing shot was filmed in my livingroom due to some script changes by the client. audio was way better in my livingroom. imagine that. we lit the second segment to match the first one as closely as possible. i used an old Lowell Omni to blast at the green screen. to say it was evenly lit would be a tremendous lie. regardless, the key pulled brilliantly in KeyLight! the background for the newscaster is the ONLY element we didn't create in-house - it's a stock 1080p background from Pond5.com.

... continued in the next post ...

 

[chris morgan]

... part two - 14000 characters written in this post! heyyyy! ...

POSTPRODUCTION


the postproduction on this was obviously the monster task. we did a first-light grade of all the lab footage in premiere, using the source settings. matching shots works pretty well if you get one in the viewer and one in the monitor.


once we had things looking reasonably good, we cut together our sequence in premiere. editing the 4K footage in real time was a breeze at 1/8 rez. we have an 8-core 3.2 ghz mac, with a GTX285 doing a little boosting for the mercury engine. had absolutely no problems working with this stuff... in fact, it was easier than my 5DII days at this stage, as we didn't have to transcode. brilliant. marty had choreographed his motions pretty rigorously, so assembling the 3D screens from his movements was actually very easy.


once we had all our VFX shots lined up, we started, as you do, with the first shot - the crane up where we first reveal the screens. selected the shot in the timeline, right-clicked, and 'replaced with after effects composition'. i cannot get over how brilliantly simple this is. coming from days of exporting footage to AE, working, re-exporting, and so on, this is a godsend. after setting the playback rez to 1/4 in AE, we were ready to get a camera solve.


this is where things got icky. loaded up the 4K footage in syntheyes, which had no problems using 4K source material. it was a bit slow, don't get me wrong, but handled it pretty well. the problem was our goddamn dolly. every little bump, blip, glitch and slip ended up sending our camera solve spinning. this ended up in me getting a crash-course in supervised tracking and optimising material for tracking. it wasn't our lack of tracking markers that caused the problem - there are more than enough contrasty points in the image to get a solid solve... it was the blurs and idiosyncrasies introduced by those little, barely perceptible bumps and jiggles. now, in hindsight, that makes all kinds of sense. but we're maniacs, you see. so to this day there are more than a few moments where the screens will jiggle or twitch due to the track not being perfect... but i had to let it go after about 65-70 hours of tracking work.


this continued to be a problem for a LOT of the shots. some solved better than others, but i've since ordered the flex-dolly track and the dolly wheels from kessler, which i anticipate will make our lives much easier in future.


another unexpected result of our brash and irresponsible image acquisition was scale. this was actually more of a headache than the glitchy tracks. exporting the camera solve from Syntheyes and opening it up in AE resulted in wildly different scales every time. and since each shot used slightly different tracking points depending on what worked for the software for that particular shot, we had no frame of reference for scale. what this ended up meaning was that most of the screen elements (which we re-used for each shot) were vastly different in size from shot to shot. actually, the screen elements were the same, but the camera's perception of the size of the lab would be vastly different. so in some shots, the "main" 'spectrum' screen would be right, in others it'd be tiny, and in others, huge. this resulted in some of the screens looking a bit more blurry and washed out - lacking the crisp, pretty lines of the first couple of screens.


this will be remedied quite simply by placement of a few key specular LED lights along the back wall, spaced evenly at 12" apart. we can of course then tell both Syntheyes and AE about that distance, and whammo, we'll have perfectly scaled camera solves every time. this will help all of our visual elements be much sharper and crisper, and save me trying to drown myself in the pool again.


once we got about halfway in to the 15-or-so VFX shots, things moved more smoothly. camera solves and scaling persisted in being the bane of our existence, but before long, everything went well. most of the screens are precomps in AE... so making buttons line up with marty's movements was as easy as finding the point when his finger would "touch" a screen, tabbing over to the precomp for that screen, and making the necessary movements and changes. i would have to say that if AE weren't as intuitive and easy to work with as it is, we would have pulled all our hair out. every time we went to do something, we just charged in thinking "well this is how it SHOULD work"... and 9/10 times, it would. that's how apple used to be. still is for iphones and whatnot, but have you tried using color? whaaaaaa.


the 3D elements like the filter housing, flat-sheet cassette model and so on were all built in their very basic forms in Cinema4D. we actually got 3D files of the filters so we were able to use those to generate the actual models of the filters (this is seen more prevalently in the filter rendering shot that is on the newscaster's screen when we return to her.)


we used Plexus and Trapcode Form to manipulate the models in the 3D environment back in AE. both of those tools are fantastic. looking back, i wish we'd taken more time to nail the "glowing" nature of the filter model when he expands the housing and carries the small rendered model, but deadlines are deadlines. we'll do more careful colouring next time. i think a 64-bit colour space and desaturating the colours in plexus would have done it.


once the VFX shots were done, we actually rendered them out of AE to 4K ProRes 422 files. dynamic link is the way we'll roll next time, but we were so eager to see things start to come together that we replaced the AE comps in premiere with rendered copies.


oh, we did all of the VFX work at 4K... partly just to see if we could, but as i mentioned in the thread in the 'shot on red' forum, we found that the VFX 'gelled' a bit better with the downsampling to 1080. the added resolution helps the final 1080 output, plain and simple. AE handled the 4K stuff like a champ. we did run in to a few plugins giving us bogus "no bigger than 2K" error messages (primarily when being used in the largely 1080p-or-smaller VFX precomps.) but we found other solutions, using old ambient recordings of oil in water and such to create monitor screens rather than fractal generators that weren't up to the task. AE handled the 4K stuff pretty effortlessly. it chugged along on some of the more saturated shots, but at no point did it become too much to deal with.


the newscaster elements were simple. the "GNS" bumper was created in Cinema4D and rendered at 4K also. we matted our newscaster out as best we could and pulled the greenscreen using KeyLight in AE, and barely adjusted the settings. the RED footage keys so easily, and this was, as i mentioned, one of the lamest, least-well-lit screens i've ever had the disappointment to be responsible for. but it doesn't show.


FINAL OUTPUT


once all our elements were back in premiere, i put a TINY bit of magic bullet looks across the whole thing - just some curves to help glue everything and give it a bit more cinematic bite, and a TINY bit of film grain to smooth out any remaining VFX wonkiness. i think i added a barely perceptible vignette as well, but can't remember.


i hate using magic bullet at this stage with tools like davinci and now speedgrade at our disposal, but i know it well, and we needed to get it done quick.


we also created the 'GNS Theme' and 'Dr. Lane's Lab Techno' to replace some temp music.


did our final export directly from premiere, and then encoded a 720P h.264 version using apple compressor (we still have to get used to using AME... i'm sure it's just as flexible as compressor, we just have to learn it.)


EPILOGUE


and there you have it! that's the breakdown of the whole shebang. boy did we learn a lot about how to be more efficient with our setups and shooting to save time in post, but overall we think it's pretty impressive for a team of 5 guys who had no right telling people we were able to do this stuff.


the scarlet has made a damn-near-hollywood-style workflow completely attainable. we've just upgraded to CS6 as well, and i'm salivating at the possibilities afforded there. also want to pick up a more powerful mercury engine card now that we can use the PC versions... need to figure that out next.


damn i just typed a lot.


love,
c

 

[Gary Paul Walker]

Really well written, thanks for sharing!
I've just finished some work keying out a greenscreen that was really poorly lit, but it keyed out really well thanks to the Scarlet (obviously no excuse for getting it lit right in the first place, but damn was it a life saver!)

 

[Uli Plank]

Did you ever try a camera track of one of those clips in CS6?

I wonder if it makes life easier regarding scaling issues…

 

[chris morgan]

thanks, gary!

yeah the chroma key stuff is amazing with the scarlet. have another project we're going to be revisiting that is based on a dual chroma-key setup that we've done pretty successfully on 5DIIs, so i'm excited to see how it works out with this beast!

uli - i haven't tried the new matchmover in CS6... will definitely be taking a look at it, though! only dabbled with some of the new premiere features and very briefly with speedgrade. AE will be next!

one other fun little trick i forgot to mention in the original post is that we painted the walls of the lab with a matte grey that was about as close to 18% as you're likely to find at home depot. then we placed 5500K CFLs behind each of the 27" mac LED monitors you see. we originally did this for aesthetic reasons, but since we light the whole affair with daylight balanced fixtures, we're able to pull an auto WB from the hotspots behind the monitors at any given moment and get pretty rock solid starting points for our grades. using the 'source settings' inside premiere is pretty flexible. a little clunky to get to - it would be nice if there was a direct contextual menu selection when right-clicking on a clip in the timeline, but there we are!

rock,
c

 

[Lliam Worthington]

Really informative and enjoyable post Chris. Thanks for sharing.

Best

Lliam