Thoughts on firmware, depth maps, editing, etc.

[Nicholas Steinwachs]

We now know that RED is working on the multiple focus depth mapping on their cameras (along with in-camera playback, which no one seems to be talking about...). I think this means a new layer to RedCode.

This got me thinking about the software possibilities in post. Imagine a bunch of 3D Scarlet/Epic Rigs lined up for a bullet-time shoot at high speed. Now, a director has full control and flexibility of his point of focus, and POV in a crazy action sequence. Essentially, you could fly around a scene at will (along a line), if editing software were to support the mapping and stitching-together of separate video streams in space to allow for full control of the picture.

So... true 4-D editing of motion pictures?

 

[Bruce Allen]

We now know that RED is working on the multiple focus depth mapping on their cameras (along with in-camera playback, which no one seems to be talking about...). I think this means a new layer to RedCode.

This got me thinking about the software possibilities in post. Imagine a bunch of 3D Scarlet/Epic Rigs lined up for a bullet-time shoot at high speed. Now, a director has full control and flexibility of his point of focus, and POV in a crazy action sequence. Essentially, you could fly around a scene at will (along a line), if editing software were to support the mapping and stitching-together of separate video streams in space to allow for full control of the picture.

So... true 4-D editing of motion pictures?

You should check out SMPTE - where Shailendra Mathur from Avid just made a presentation on light field editing

http://www.cvent.com/events/smpte-2...stom-17-c431a68b25ee45ec8cb988e62d705e26.aspx

"Editing and Managing Multi-Channel Video - A Light Field Approach"
Shailendra Mathur (Avid, Canada); Steve McNeill (Avid, Canada)

A strong trend in the media industry is acquisition of higher data resolutions and larger ranges in color, spatial and temporal domains. The data is in the form of multiple views over these domains. It may be captured from multiple devices, a single device with high resolution/range capabilities or, a synthetic source such as a CGI scene. The goal of a production has now become to capture as much information as possible in the production assets for creative freedom i.e,. "super-sample the world!" and extract only the views needed for mastering. The role of traditional story telling tools becomes challenging in applying well known editorial and data management techniques to the variety of sources and, forms of multi-view data representations. To create an editorial and data management system to work with these multi-view sources, concepts from the area of Light Fields are used. A data model and a run-time system is proposed that allow the various view data sources to be grouped in a way that they appear as a single light field source to the editing and data management functions. Using output view requirements and using key meta-data defining the inter-relationship between the multiple sources, new output view information can be generated to be used in the editing and management functions. In particular, the importance of using range information in each domain is shown to be key to extend common editing and data management techniques to multiple view sources.

Presenter bio: Shailendra Mathur is the Director - Chief Architect of Professional Editing Applications at Avid Technology. Avid is an Oscar, Emmy and Grammy award winning company that creates the digital audio and video technology used to make feature films, music recordings, television shows, live concert tours and news broadcasts, to music and movies made at home. Shailendra obtained his Master's degree at the Center of Intelligent Machines, McGill University, with a thesis on estimation theoretic frameworks for visual reconstruction. His research and engineering interests have led to publications and patents in the area of computer vision, medical imaging, visual effects, graphics and media processing architectures.:

Bruce Allen
www.boacinema.com

 

[Nicholas Steinwachs]

It would be great to hear his thoughts on data management and light fields. Unfortunately, I live in Massachusetts.

In any case, the way I see it, you'd only need two or three key elements in the scene easily discernible by software, and spatially relate them to each other, and to other viewpoints, so the system can streamline the editing. You already have time data with which simultaneous cameras are related, and now light field information.

I am curious how light fields are defined, captured and encoded. Hopefully there's enough information online to do some Google research.