DISNEY DEVELOPS NEW METHOD...

[Jeff Kilgroe]

Erm.... Nothing new here. Apparently the only "news" is that they have developed some software to assist, or even mostly do, a common task within the VFX biz. The common spatial seam between elements in video clips or even elements within a single clip or image is a widely explored property. The big thing at Siggraph several years back was to combine mediospatial detection within images with automatic referenced cloning to make arbitrarily scaled images. So if you had an image of a beach sunset with a large rock on the left of the frame and a palm tree on the right, sun somewhat centered in the image, the software could scale the image without distorting the individual elements, just by filling in the non-detailed areas. Unfortunately people didn't go for it, no one wanted to have web sites, blogs, whatever, changing their images on the fly just to fit some arbitrary user's screen or window space.

Same thing for taking multiple clips and combining to change placement or putting two actors' best performances together in the same shot... Not a new thing. Just seems like a new tool to do it. For low-budget stuff, I agree with other sentiments above. It's better to keep rolling until they get it right. Technological solutions for artistic problems are sometimes not the way to go...

 

[Elsie N]

...

Same thing for taking multiple clips and combining to change placement or putting two actors' best performances together in the same shot... Not a new thing. Just seems like a new tool to do it. For low-budget stuff, I agree with other sentiments above. It's better to keep rolling until they get it right. Technological solutions for artistic problems are sometimes not the way to go...

Jeff,

As always, a well researched and very readable explanation of the subject matter. And while I agree with the idea of shooting however much it takes to get the subject covered (one reason I wanted two cameras is to ensure I have excess footage of each and every scene) I also realize that there will be times when re-framing in post to change a scene may not do the scene justice. That is why a plug-in with the Disney solution that would allow me to take a clean portion of a landscape and seamlessly blend it in with the central focus of the scene is very appealing to me.

Not only that, but time and time again when I get into post I see something in a scene I wish I had noticed before and corrected. It would really be nice to have something that apparently will easily allow me to correct the flaw. This could include pasting parts of an actor's scene together to get the best total performance. At times that may be the only way to get a good performance if you are using untrained actors. I understand on big or medium budget productions you just re-shoot the scene entirely, but for those of us who are neither, sometimes a technological solution is the only way to go.

Of course this may all be moot. There's no evidence that I know of that Disney will make the method available. '-(

 

[Bruce Allen]

Elsie... listen to Marc and Jeff.

I'm also not sure what's so expensive, "Old Hollywood" or elitist about:

1. shooting until actors get it right
or
2. combining two clips using existing roto and tracking tools

Sure - new methods (such as this current one, which is similar to a lot of other research) will pop up, and often be very useful... but there will be caveats for sure.

Personally, what I think is expensive is leaving the location incorrectly assuming that you can stitch stuff together.

Then you find out later in post that the algorithm doesn't work on that particular shot.

Then you have to reshoot not just that shot but the whole scene because the location is no longer available, the actors aren't available, etc. and you need everything to match.

Bruce Allen
www.boacinema.com

 

[Elsie N]

...
1. shooting until actors get it right
or
2. combining two clips using existing roto and tracking tools

Sure - new methods (such as this current one, which is similar to a lot of other research) will pop up, and often be very useful... but there will be caveats for sure.

Personally, what I think is expensive is leaving the location incorrectly assuming that you can stitch stuff together.

Then you find out later in post that the algorithm doesn't work on that particular shot.

Then you have to reshoot not just that shot but the whole scene because the location is no longer available, the actors aren't available, etc. and you need everything to match.

Bruce Allen
www.boacinema.com

Bruce, I actually do get it that it's better to get the scene right, on camera. But I think where you, Marc, and Jeff are misunderstanding me is that I want something that will cover my ass in post if somehow things aren't possible on set.

The example I keep harking back to is the shot where the Daphne Duck is making its way across open country littered with burned out mesquite bushes. I've already shot the footage... over a year ago after the extreme drought and brush fires in the area with this scene in mind. At the time the Daphne Duck wasn't available and of course, the actors haven't been cast who will ride in the open air all-terrain vehicle. When shooting begins, I will shoot the vehicle traveling over this same route.

Problem is, since I first shot that background... oil wells, high lines, caliche roads, yada, yada yada now pepper the landscape. This stark scene is important to the movie as this is a post cataclysmic movie and this particular scene brings that home to the audience. It is the previously impenetrable membrane that separates two of the isobars of bifurcated civilizations.

As I understand the Disney release, their method will allow me to shoot that scene in the same place and will then afford me the means to place those different shoots together. And, while I will attempt to get the new shot done at approximately the same time of year and time of day and the same meteorological conditions, even if I'm off a little on exposure, IIRC the Disney app will afford me the opportunity to match those conditions.

Yes, I understand there are other ways to do this already and for those who spend most of their time doing post work, it probably seems pretty easy. But I'm involved in the entire process of the movie and have to devote time to all aspects of that so I'm looking for a quick and easy, yet professional-looking way of solving any problems.

Disney has done the work that I believe, whether you agree or not, will meet my needs.

 

[Bruce Allen]

Here's my prediction: it won't work for the shot you described without substantial additional VFX work (all of that keying / roto / matte painting stuff you seem to not want to learn).

One big issue is that you are trying to combine two different backgrounds (old pristine one and new one with unwanted bits).

When you have your hero vehicle driving in front of the background with unwanted oil wells, high lines, etc... how is the computer going to know which bits are hero car and which bits are new unwanted background bits? Do you think it will be able to cut them out perfectly with motion blur etc?

The algorithm seems to be good at matching movements - but not that sophisticated at cutting stuff out when you're trying to combine two shots with different backgrounds.

Also... take a close look at the window shot here:
http://zurich.disneyresearch.com/~owang/pub/ducttake.html

It doesn't quite line up (you get a ghosting effect). It's not that visible at web resolution but will be distracting if you're planning to master at 1080p or (heavens forbid) 4K. This ghosting is because of perspective / parallax issues.

Basically: the Disney stuff (or other tools like it) might help but you're going to need to learn traditional VFX techniques anyway to fix the inevitable issues that it won't fully solve or the remaining artifacts that it will create as a byproduct.

Bruce Allen
www.boacinema.com

 

[Elsie N]

Here's my prediction: it won't work for the shot you described without substantial additional VFX work (all of that keying / roto / matte painting stuff you seem to not want to learn).

...if you're planning to master at 1080p or (heavens forbid) 4K...

Bruce Allen
www.boacinema.com

Heh, heh... yes, mastering at 4k. '-)

Like I said before, there's no evidence that Disney will make this available outside their own circles... but if they do, I would like to try it in order to see if it works for my needs. If not... well, a RAW edit is after all, non-destructive.

 

[Björn Benckert]

I don't see anything about this being pitched to low-budget movies. This is a technique where you stitch the left side of one good take with the right side of another good take, and composite the two together in a way that's imperceptible. My memory is that George Lucas was doing this with Phantom Menace back in 1999, back when this was expensive and time-consuming.

To me, the low-budget way of doing this is to keep shooting until you have a scene where both actors deliver good performances at the same time. This can be achieved with lots of rehearsals, good planning, and good acting -- and zero effects. Sometimes, I think too many people look for technological solutions for artistic problems, when I think the natural approach (doing it right in the first place) is simpler and much cheaper and faster to do. Really, how much would five more takes on the set really cost, compared to a 2-minute seamless composite?

you do not do much motion control stuff I suppose or crowd dup scenes or A B camera stich takes etc.. I'm really impressed if you can make a good living by only delivering straight shots... But for the rest of us there is quite a bit more hassle involved before any kind of larger sums of money reach our accounts

Basically it's a tracking tool that uses similarities / differences in two shots and sample them together with different transfer modes and also apply 3D tracking VR camera adjustments... Which happen on a daily basis in every post house nowadays and it's quite time consuming, manually 3D tracking the two scenes project them onto 3d planes / models and then roto / spline the two takes together. This tool kind of does all that in semi automatic way. Giving you great result with shadow overlapping etc. And film making today for most of us is far more than getting good takes and edit them together, as things have evolved even split takes are now whats done by "soccer moms".

I wish I was aloud to show you the result of my 24:7 struggle that has been going on for a month time now.. but it's not out yet. 2 minutes of motion control with I do not even know the amount of layers, splines, keys and so on that has been pushed into it. But the duct take stuff would have giving me and my team lot more sleep lately, and people ask for the service and pays quite heavily to have it done so I'm sure disney will sell more than a few copies of what they have done. Even though others will quickly follow and make simular apps.

My guess is that they made it as a nuke plugin. It would be most logical. Also Autodesk is already onto something simular, their iphone app called 123D catch (thats for free much worth downloading) use some of the same math.


The funny thing is that this stuff probably makes it into final cut XI and iphone 6 before any of the big player get it out

 

[chad vachter]

i go onto reduser for the first time in a few weeks, and the very first post i read , which was a interesting read ....

someone attacks the person for grammar and spelling , next ... several people singularly explain how filmmaking works,, and how
this persons interest in technology , is some sort of offense to their superior thinking ...

i know the business is slow , and rates and kit rentals are definately not what they used to be... but try and keep mean vibes to oneself

some folks really love this craft , and the journey to learning is just that .. a journey

i dont think any of you would put out the candle of some person on their journey to enlightenment ..purposely ... would you ?

.. i learn everytime i read a post from jeff kilgore or bjorn benckert, both you guys are irreplaceable golden assets to this forum .
and there are many other very knowledegeable folks on here as well.

thanks for posting this elsie ,
ps i was in no way was eluding to you being new or just starting your journey

pps my grammar and punctuation is terrible , i know this and care very little about it. so spare me any lessons
thanks

 

[Elsie N]

i go onto reduser for the first time in a few weeks, and the very first post i read , which was a interesting read .... someone attacks the person for grammar and spelling , next ... several people singularly explain how filmmaking works,, and how this persons interest in technology , is some sort of offense to their superior thinking ... i know the business is slow , and rates and kit rentals are definately not what they used to be... but try and keep mean vibes to oneself some folks really love this craft , and the journey to learning is just that .. a journey i dont think any of you would put out the candle of some person on their journey to enlightenment ..purposely ... would you ? .. i learn everytime i read a post from jeff kilgore or bjorn benckert, both you guys are irreplaceable golden assets to this forum . and there are many other very knowledegeable folks on here as well. thanks for posting this elsie , ps i was in no way was eluding to you being new or just starting your journey pps my grammar and punctuation is terrible , i know this and care very little about it. so spare me any lessons thanks

hi Chad, thanks for the simpatico post but I feel you have gotten the wrong idea about the way some of the posts sound when read.

For instance, Mark Halfich and I have had an ongoing friendly "feud" for some time now. I've said much worse to Mark than he did to me. '-)

As for Jeff, Bruce, and Marc... I consider their posts as healthy skepticism. Bjorn joining in on the other side of the discussion really helped balance the two schools of thought and while I was in no way dissuaded from my earlier stance, Bjorn certainly reinforced my faith in that stance.

Very kind of you to say you do not see me as being new or just starting my journey, but since my next feature movie will be my first one, I really do resemble that status. However, I've been involved in shooting enough things that really amounts to just practice so that I am not lacking in confidence going forward. I owe most of that to what I've learned on this forum.

And it's great to see time and time again when posters like yourself step forward to weigh in on what appears on the outside as being an injustice. Redusers got each others back!

 

[Bruce Allen]

The point that I made and that Bjorn also makes is that this algorithm may help save some time. But you still need to learn traditional VFX techniques as they are extremely relevant.

What confuses me is why the inexperienced folks are so disdainful of plain old manual labor and teamwork. Also they really think that professional VFX artists are stuck in the mud and actually hate indies or love roto or something?! I love new stuff. I'll be at SIGGRAPH. I love light field tech, photogrammetry, new tricks, etc. I've helped out a ton of indie guys.

At the same time, just be realistic: you're going to need a team to do VFX shots. Or become good at VFX yourself. Working at 4K with RotoBrush or DuctTape might not get it done faster or better than learning to roto.

In fact all of this high-tech stuff is often a bit of a trap. It's what companies use to get you to upgrade to their new product. Not saying that's always the case, of course.

If you aren't experienced... the best way to look at a project is to estimate it assuming that all of the fancy things will fail.

Just look at that shot and say "hmm, that's a couple days of roto". Think of yourself (or your friend) doing that roto. Then try to do it differently if you can. Or if you must... accept the roto. Because your shot is that awesome that it DESERVES days of work in post. Just make sure the shot is awesome then!

The sad bit is when you set up a project, completely underestimate the time to fix it in post - and end up never finishing it. I totally speak from experience on that one

My guess is that they made it as a nuke plugin. It would be most logical. Also Autodesk is already onto something simular, their iphone app called 123D catch (thats for free much worth downloading) use some of the same math.

Agree... especially since one of the videos has an A-B comparison with F_Align!

Bruce Allen
www.boacinema.com

 

[Jeff Kilgroe]

If I understand what Disney has here, based on the generic description, it seems like it's a one-click approach to a very common VFX task. Essentially, you have actor A on the left side of the frame and a good take of that as "clip 1". You have actor B on the right of the frame and a good take of him is "clip 5". So in this tool you bring in clip 1 and clip 5 and set a quick and dirty divider line between the two actors. The software then automatically tracks and/ or rotos as needed to find an arbitrary median between the two clips where it can merge the two with the least amount of spatial or temporal distortion. Really, it could be very useful, it could turn a 60 to 90 minute process for a single shot into a 3 minute click and done process.

This feature coming to Adobe Premiere and/ or AE in 3, 2, 1....

 

[Sandro Klopfstein]

looks like the f_align plugin that comes with the foundry's furnace.

 

[Marc Wielage]

Really, it could be very useful, it could turn a 60 to 90 minute process for a single shot into a 3 minute click and done process.

I can remember a time when if we had to manually track a power window on a subject, it might take us the better part of 45 minutes or even an hour to painstakingly notch in dozens of key frames and go over it again and again to make sure it was absolutely undetectable. And now tracking takes maybe 10-15 seconds and is pretty much automatic, using about 100 tracking points (in Resolve or Baselight).

So some things do get faster and are easily done with modern software. In a case like this, though, I'm not convinced combining 2 live action shots is going to be practical for most people at modest budgets. It'd be a lot better just to get it right in the first place.

If the o.p. is just trying to key a person into an existing background, green screen (with a lot of planning and tests) can be extremely effective, even in a moving shot, particularly if you shoot tracking markers on both shots and also have a reflective ball and a gray scale ball for light reference on the live action. All of that stuff can be used and measured by the VFX crew to get the lighting exactly right in the final composite.