Virtual Production - A Cinematographers' Conversation

[AJ Young]

The ACS released a fantastic video on virtual production. To quote Tom Gleeson of the ACS Technology Committee:

In August, NEP Studios and Spectre Studios worked with the ACS to set up a pop-up version of the studio. Under the guidance of the incredibly talented Bonnie Elliott, ACS, and Kieran Fowler, NZCS, ACS, we photographed a series of scenarios exploring the new process of virtual production using large-scale, computer-driven LED walls. The ACS Technology Committee has created a video of the event discussing the basics of virtual production and specifically its use, challenges and impact for cinematographers.

Personally, this technology is going to get cheaper and easier to use. We can already use Unreal Engine and Blender for free.

What does everyone think?

For those interested in learning virtual production:

Unreal Engine has some good courses to get you started: https://www.unrealengine.com/en-US/onlinelearning-courses

Blender is also a popular program that isn't yet used for real time virtual production, but punches well above its weight for all other VFX: https://www.blender.org/

However, there are people starting to experiment with Blender as an alternative to Unreal Engine: https://youtu.be/TK6FeXOCwQI

 

[Karim D. Ghantous]

I would not use the term 'virtual production' for this process. I mean it's a very advanced form of rear projection or green screen. But virtual production is when everything is CGI, IMHO. IOW, animation.

 

[Med D]

Very interesting final line in the narration. This space is so dynamic right now. I'm looking at a design for an indy studio using LEDs and it is surprisingly easy to do. Of course there are a bunch of middleware and hardware suppliers trying to edge into the middle to make some money off of this new tech, but your observation that Unity/Unreal and Blender, plus a pile of LED screens, is really all that is needed. So far I have not seen a DIY tutorial for a budget studio but I suspect there will be one within weeks, something that raids Costco for panels and uses off-the shelf mounting and cabling. I expect this to completely re-map the human resources side of production, not just the number of people needed but the tasks they perform.

I did not understand the shot with the guy shooting handheld totally within the virtual space (not the LED space). It's very easy to animate a camera in the game engine to accomplish what I think is the same thing.

I really expected the two DPs to be revealed as being on virtual sets at the end. Each of them look like they were in a virtual space, I mean the film camera in the corner screams Unity Asset Store.

Karim, the terminology is already out there. I understand your point but I don't think it will change.



Check out our virtual version of the famous Kalalau Trail on Kaua'i:
The Kalalau Trail from CraniaLab

 

[Michael Tiemann]

Incredible vision from the ACS that does not require drinking a tremendous amount of Kool-Aid. I'm impressed!

 

[Tom Gleeson]

Oops I just started another thread announcing the video and I did not realise AJ had already posted. Karim Virtual Production can take many forms and yes the totally CGI generated world version is the most "pure" but I think Mike Seymour sums it up best at the 4min 22 second mark. Med D I am not sure I fully understand your point but the use of humans operating the virtual camera creates an "organic" and realistic movement that is actually quite hard to animate. The way a human operator responds to events and even the small errors all help to create a more authentic feel. For the same reason motion capture is used to animate digital characters. Sorry the DOPs were interviewed in my living room and that is a real 1939 Mitchel camera* behind them but I would not be surprised if in fifteen years you could not have entire walls that are light emissive and the room could track your head position and the wall could be a visual portal to any view in the world. Apple TV screensaver on crack ! (calling Phil Holland)

As AJ points out learning the basics of this process could become a very powerful tool in your future career ahead.

* No it was not my first camera. I'm not that old ;-)

 

[Med D]

I understand the "organic" look, but that is a massive amount of hardware in both $$ and kg just to capture an entirely virtual view. Seems super-primitive. I'd go for an animated camera first but I am on a budget. Animated cam also leaves you with something more readily modified if needed later.

 

[Tom Gleeson]

Med D yes it is an expensive business building a LED Volume studio and with the technology develping quickly it is also a brave choice. BUT once a studio is built the cost is shared by multiple productions. On Monday you can walk in to the Studio with a digital file of Ancient Rome shoot your gladiator fight Tuesday then Wednesday another production bumps in and they upload their Amazon Jungle file and Thursday we are shooting in a steamy Forest. These studios will probably operate as independent entities with multiple customers with the cost being amortised over many productions. But there is no doubt that building a LED volume would be a gamble as there is no business tougher than show business.

 

[AJ Young]

I think we're underestimating how fast and cheap this tech will evolve. The biggest financial hurdle are the LED walls and tracking the real camera, but the software will only streamline the process more and more.

Global illumination in the virtual environment is already outstanding, especially with what UE4 and Blender's Eevee do with baking in the lighting onto the models. Not everything needs to do ray tracing and game engines take advantage of visual tricks to simulate real world lighting (ie ambient occlusion). There will be more tricks like the above that can free up the GPU to process more photo-realistic imagery.

Here's an example of baking lighting via Blender's Eevee:

 

[Med D]

Sorry I only meant the part of using a real cam op + tracking to shoot a virtual handheld view was "expensive". The rest of the studio I think is well worth the cost. And AJ I agree the costs will soon be such that indies will be using this tech everywhere.

 

[Karl Janisse]

For anyone interested in joining in, I'll be hosting a panel discussion for the Canadian Society of Cinematographers on Virtual Production tomorrow at 2pm EST/11am PST. Check it out here: https://www.youtube.com/c/CanadianSocietyofCinematographers.