Scarlet - makes VFX/post faster?

[Laco Gaal]

So, I saw that the Scarlet will include a 3-axis motion sensor, and I though if this thing works, it will spare me a lot of hours of tracking. This would be really amazing.

But my question is, that in the case of the Fixed Scarlet:
- would it be possible to also 'record' the current focal length to the r3d?
- would it be even possible to record the focus distance, continuously?

Juts think of it: you import your Scarlet footage to a compositor application, from the 3-axis sensor, you will get the exact same virtual camera movement, and from the focus distance you would get the same virtual focus pulling. This means that if you put a 3D/2D object in your comp, then it would behave just like it would have been there at the shooting, amazing!

 

[Steve Wake]

I'm interested to see what RED or a third party like the Foundry do with that motion sensor info too, but more having to do with image stabilization.

As far as info in the metadata, Jarred confirmed many moons ago that the motion sensor data will be carried in the frame-by-frame metadata. I'd be surprised if the other info you mentioned is not also there.

Something else on VFX wishlists is a Z-channel, described here:

the french guys keep talking about this depth channel thing... that would be something. there was a company making that, but they sold it to microsoft who implemented it as part of their kinect program... i'd love to have a good Z channel to work with.

for those that don't know what it is, a depth channel is a greyscale image representing distance from the camera. for example, a person in the foreground will appear dark, and the background white (or vise-versa). you can use this to begin building a matte. depending on the quality, this is at least a helpful place to start for stereoscopic 3D work (especially when done in post), just about any visual effects shot (rotoscoping/keying), and you could also do things like make your depth of field more shallow (but please don't ever do that just because you can, that's not going to make it "look like film", it's just going to make the background look blurry).

in any case, i'm excited to see what RED has in store.

 

[paulherrin]

nice reference...

nice reference...

I actually haven't looked into what's going on with the 3-axis motion sensor... sounds like there's some potential there. Tracking and Stabilization are essentially one in the same, stabilization is just one route for using the tracking data. There are issues that are more difficult to solve in post, such as motion blur induced by the much avoided "earthquake cam". You are very optimistic about the motion sensing, that would be awesome! But realistically, I'm not sure it will wipe out the need for motion tracking altogether, even if it's pretty good. Luckily, the software is getting really good for tracking, and the resolution and quality of pictures is going up, so that helps. I mostly use boujou for tracking, and generally get good results. I also use NukeX to track simpler things, and for doing stabilization. I also have PFTrack, which I keep hearing is better than boujou, so I'm going to start checking it out some more.

Yes, the focal length is stored in the metadata, and I don't see why the focus distance wouldn't be as well. All the electronic lens data should be stored, plus a million other awesome things. Very helpful, and that saves A LOT of time and hassle to gather that data on set. Awesome.

Anyways! In addition to a really plump metadata, I think they help speed up vfx/post work in other ways as well. For example, it takes a lot less time to get the high quality color correction/grade you want - the amount of data stored in an r3d is mouth-watering. Also, the measured resolution helps a lot of things, obviously.

Plus, especially for tracking or initial, pre-motion blur rotoscoping, maybe some keying - I think it will be a popular idea to record HDR. That will give you a faster shutter speed of the same frame, thus reducing motion blur, which is essential to tracking/stabilization as well as pulling mattes or keys. I know I'll be testing all this stuff out...

As for the Z-Depth stuff, I haven't had time to really do any research or development, but I've heard that you can modify some consumer cameras (like the kinect) to record something (not sure how usable it is). I'm also not sure if RED has done any heavy R&D on that, though I would hope that would come about in the future. It would be EXTREMELY helpful for the people that do all that super tedious post-production stereoscopic stuff, where they paint their own depth channels. I hope I never have to do that... Also, it would help for techniques like projection mapping, especially on complex geometry, like a body or even a face. If I get some time to do some experimentation, I will, and I'll definitely share what I learn.

Exciting stuff.

 

[Leo Sardello]

I think the additional information provided by the 3 axis sensor, the lens information and use of the x frame from HDRx™™ will prove to be a powerful combination for visual FX. I can’t wait to see what starts to come from all that gooey metadata!

If I remember correctly it was stated that the 3 axis info would be in the metadata and the SDK- but would require a third party utility to utilize. I know I’m totally poorly paraphrasing here, but in other words, the info is there but it’s up to someone else to figure out how to make use of it (and that someone could be you, or me, or anyone). The point is I don’t think you’ll just be able to import the data into any compositing program at launch and say “there, that’s where my camera was”. Of course I could be way wrong, that’s just the last thing I remember being said about it.

One thing I’ve missed is whether the EPIC has that sensor as well. If it does it might be worth tossing a question out to one of the *lucky* early users to see what they’ve gained from it.

The Kinect projects a continuous infrared “grid” that is read back by a CMOS sensor to obtain its depth information. It’s a sweet simple concept and is very cool and I’m sure we will see all kinds of similar tech attached to who knows what in the future. Hacking a Kinect and finding a useful way to use that information is a popular hobby right now. While I’ve seen some very impressive stuff, it doesn’t really seem to currently have any practical applications for VFX work. I’d love to see what some people come up with in the future for it though. In theory there’s tons of stuff one could do with it, but you know, I’m just saying- there’s a big difference between everyone being able to use that data in say AE, and some FX team hacking them for a specific use on a big set.

 

[paulherrin]

with the foundry's involvement with RED, i have a feeling nuke will soon be supporting some built-in options for reading that data. if it doesn't... i'll be doing some scripting.

 

[Laco Gaal]

with the foundry's involvement with RED, i have a feeling nuke will soon be supporting some built-in options for reading that data. if it doesn't... i'll be doing some scripting.

Well the best thing would be if the current top of the line 3D tracking applications could understand these data. Boujou, and PFTrack does an incredible job tracking footages, with these datas, tracking would be more precise, and faster.