Welcome to our community

Be a part of something great, join today!

Register Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

Adventures in Virtual Production

A

AJ Young

Active member
Joined
Aug 8, 2019
Messages
33
Reaction score
0
Points
0
Location
Los Angeles, CA
Website
www.ajyoungdp.com
Since the industry shutdown all of my live action work, I decided to learn a new skill set and dived deep into the world of virtual production. I said it in college and still believe it today that narrative film will increasingly move towards virtual production. Big budget films already utilize a ton of CG environments and the Mandalorian showed us how incredible real time rendering on an LED wall can be.

As a primer, I found this doc from Epic games gives a fantastic intro to virtual production: https://cdn2.unrealengine.com/Unreal+Engine%2Fvpfieldguide%2FVP-Field-Guide-V1.2.02-5d28ccec9909ff626e42c619bcbe8ed2bf83138d.pdf

Matt Workman has been documenting his journey with indie virtual production on YouTube and it's becoming clear how easy it is for even the smallest of budgets to utilize real time rendering. I wouldn't be surprised to see sound stages or studios opening up that offer LED volumes and pre-made virtual sets at an affordable price for even micro budget features.

Here's what I did so far:

render-comparison-06022020001.thumb.jpg.16cf02039673281e66dd8be46b47380e.jpg

Attached are renders from three engines: Blender's Eevee, Unreal Engine 4, and AMD ProRender. (please excuse the heavy JPG compression, shoot me a DM for an uncompressed version)

I made the building meshes from scratch using textures from OpenGameArt.org and following Ian Hubert's lazy building tutorials. The man is a 3D photoscan from Render People. All of these assets were free.

Lighting wise, I was more focused on getting a realistic image over a stylized one. Personally, the scene is too bright and the emission materials on the buildings need more nuance.

  • Blender's Eevee:

    Honestly, I'm beyond impressed with Eevee. It's incredibly fast. For those who don't know, Eevee is Blender's real time render engine. It's what Blender uses for rendering the viewport, but it's also designed to be a rendering engine in its own right. Most of Eevee's shaders and lights seamlessly transfer over to Blender's Cycles (their PBR engine).
  • UE4:

    After building and texturing the meshes in Blender, I imported them into UE4 via .fbx. There was a bit of a learning curve and manual adjustments for the materials, but ultimately I was able to rebuild the scene. The only hitch were the street lamps.

    In Blender, I replicated the street lamps using an array modifier which duplicated the meshes and lights. The array modifier doesn't carry over the lights into UE4 via the .fbx, so I had to import a single street lamp and build a blueprint in UE4 that combined the mesh and light.

    In the attached image, the street lamps aren't in the same place in UE4 because I was approximating. My next step is to find a streamlined way to import/export meshes, materials, camera, and lights between Blender and UE4. I believe some python is in order!

    As expected, UE4 looks great!
  • AMD ProRender:

    Blender's PBR engine, Cycles, is great. However, it only works with CUDA and OpenCL. I currently only have a late 2019 MacBook Pro 16". It's GPU is the AMD Radeon Pro 5500M 8GB. Newer Apple computers only use the Metal API, which Blender currently has no support for.

    Luckily, AMD has their own render engine called ProRender. Needless today, the results are great and incredibly accurate. This engine isn't a real time. Render time for the AMD shot was 9 minutes. This render is definitely too bright and needs minutiae everywhere.

My final thoughts:

Even though this is a group for Unreal, I'm astonished by Eevee, particularly how incredibly fast it is for running only on slow display GPU. (Again, Blender has no support for Apple Metal, so it defaults rendering to my Intel 630 GPU which is for displays)

The next iteration of Blender will be utilizing OpenXR. According to the Blender Foundation, they'll slowly be integrating VR functionality into Blender and version 2.83 will allow for viewing scenes in VR. With that in mind, I'm definitely going to experiment with virtual production inside Blender.

As for Metal support, I believe Blender will be moving from OpenCL/GL to Vulkan in the near future. From what I've found, it's easy to translate from Vulkan to Metal. (This part is a bit above my head as a DP, so I'll just have to wait or use a Windows machine)

Does anyone have any useful guides that have a streamlined process for moving things between Blender/UE4? I'm from live action and love how easy it is to bring footage from Premiere/FCPX to DaVinci and back via XML. Is there something similar or is .fbx the only way?

Correction:

After further research, I discovered that Eevee can only run on the GPU (according to Blender's documentation). Eevee is in fact running on my GPU on my laptop, but it is running on the display GPU (Intel UHD Graphics 630 1.5GB). My MacBook Pro has 2 GPU's, one for processing graphics and one processing displays.

Regardless, impressive.
 
Last edited:
From watching the Ue5+playstation 5 demo I have not seen such progress in a long time in games. Cant wait to se how the UE5 engine will shake the industry.
 
I don't know much about UE, but Unity works with native Blender files. It's actually very amazing to me, I have .blend files in my Unity project folders and can edit them in Blender and the changes turn up immediately in Unity. This is really useful for things like UV editing, scale, pivot points etc. I recently learned how to use level-of-detail structures directly in Blender as well. These turn up in Unity with LOD support already in place. For a free piece of software, Blender is so good that it makes one wonder what Adobe does with all the money they make.

I have a massive Unity project about to release. It's a virtual version of the famous Kalalau Trail on Kauai. I released a teaser version a year ago but this is the full eleven miles of trail. I did it mostly as a learning experience but it became something else. I am very fond of both Blender and Unity.
 
Last edited:
AJ,

Congratulations on immersing yourself into Virtual Production. Skills learnt working with the render engines building environments will greatly improve your communication depth and understanding when working on a Virtual Production.

At the ASC we made this video looking at Virtual Production specifically through the lens of cinematography

https://youtu.be/3g_GhufsQ7E
 
Very Informative - Thanks for the insights

Very Informative - Thanks for the insights

Here in Portland, OR things hit a halt during the pandemic and three local stages have popped up with these setups since. Have had the opportunity to play around with it as they were being setup and first marketed to the very small soundstage presence in Portland. Thanks for the links and illumination here filling in some of the gaps. Has been a mixed bag so far, but see the great potential in it with enough computing power and knowledgable techs who specialize (also still developing here)
 
In Norway's South East there have been some really exciting developments into VP stages (much like the Stagecraft for the Mandalorian). I have been talking with some people here about it and it is really a hot interest. There is still a long way to go, especially with the content creation and the work force needed to drive projects. Larger film budgets will probably take the most advantage of this, but I am excited to see the smaller projects, maybe music videos bring out the creativity. I have been getting into Blender, as of last week, so I can learn and understand more of how VP works.

It's really exciting stuff ~ but for a lot of people, midblowing and baffling. :D
 
Great thread. I started with Blender 2.8 last year with an intent to apply it to pre vis, and of course that lead me to Unreal engine. It was an exciting experiment. You've gone much farther than I did, AJ. I found Maya to be a much more intuitive interface, but although it has nothing to compare with Eveee, it's included Arnold renderer is worth the price of the program, now just $300 per year for the indie license. For what I'm doing I have no need of UE4, but am certainly keeping a close eye on Blender. An astonishingly powerful program that keeps getting better.
 
You must log in or register to reply here.
Back
Top