In camera stabilization idea. What am i missing?

[Jacob Callaghan]

Hello REDUSERs,

First time posting here, so hello out there!

I had this thought the other month of an in camera/post stabilizing system.

So imagine you take the red dragon shooting 6k and add sensors for every axis the camera can move. One for yaw, pitch, roll and so on. These sensors record all this information into metadata telling exactly where the camera moved in each shot. You then take this metadata and put into a stabilization software that can read this metadata. (no software exists out there that does this that i know of) Then using the 6k files you post stabilize the image using the metadata. Since you used the metadata there is no warping no motion tracking no nothing. It is an accurate stabilized image based off of what the sensors on the camera recorded.

So if we shoot 6k with the ultimate goal of stabilized 4k footage we must have a way to monitor 4k while recording 6k. So if on our camera monitors we saw the 4k but were recording 6k that would allow for proper framing.

Now i think this idea makes a lot of sense with not to many down falls. Obviously you would still need to get a somewhat stable image to begin with but what am i missing. Why would this not work?

Thanks,

Jacob

 

[Gavin Greenwalt]

I've done some projects like this and this would work and I do it but it's not a good solution.

1) Motion blur. You would have to shoot at around 1/200th of a second to eliminate nodal motion blur. This would mean you would have to rate your camera at 1/4 of the speed compared to 1/48th 180 degree shutter. So if you rate Dragon at 800 ISO dragon becomes 200 ISO. If you rate Dragon at 320ISO you would have to rate it at 50 ISO relative to a 180 degree shutter. The next problem is that without motion blur you don't only eliminate nodal rotation motion blur but you also lose camera translation motion blur, intended nodal motion blur as well as object motion blur. Deliberate nodal motion blur is trivial to re-add since it's global to the frame. Object translation motion blur and parallax motion blur is really really hard to reproduce. Imagine rain as an extreme example. In one frame a rain drop is there... in the next frame the raindrop is gone. There is no way for a motion estimation algorithm to determine which rain drop is which and you end up with flickery rain. Another extreme example would be a window. Imagine someone drives by the window. The motion estimation algorithm would detect the motion and blur those pixels. The end result would be a blurry reflection... but it would also blur the face of the person behind the window. The only true solution to the problem is something called Deconvolution. Using the gyro data you can theoretically remove just the nodal blur but leave in all of the existing motion blur. The problem is the word "Theoretically", in practice deblur/deconvolution processes lose tons of resolution (It would for for 640x480 SD from 4k) and they introduce tons of artifacts and ringing. All in all there are going to be a ton of sacrifices.

2) Lenses. Lenses aren't terribly uniform. They look pretty good but once you throw up a grid you can see how much sharpness and warping there is as you near the edges of frame. Chromatic aberration is far worse at the corners than it is in the center for instance. You can remove some of this but again it comes at a resolution and artifact cost as you correct for CA, Vignetting and geometry.

3) Rolling Shutter. With a camera like Dragon the rolling shutter isn't noticeable most of the time when shooting but once you eliminate all of the distracting motion blur you'll suddenly see the frame start to jiggle and warp like you were shooting through an old fashioned glass window. Theoretically removable but again emphasis on theoretical.

4) Perspective. Even with a teeny tiny example of this, stabilized lenses you can see strange artifacts with large motions being stabilized. This isn't really noticeable with long lenses but if you shoot video with a IS'ed wide angle you can see the vanishing points going all wonky as the image moves around. Again, this is correctable with software but with artifacts blah blah blah.

5) Lens Flares and bokeh. Lots of lens artifacts vary from corner to center. For instance bokeh often starts to "cat eye" at the edge of the frame. Imagine panning and scanning around on that image. The bokeh of an object would be morphing and transforming which would be very distracting.

Lens flares also distinctly go from the light source towards the center of frame. They're a radial artifact from the bounced line. If you stabilize your lens flare will now start "bouncing around". To see a great example of this, watch the opening credits to the TV Show House. They shot and stabilized some aerial footage and the result is an obnoxious jittery lens flare.

So in conclusion yes it would work but it's stupid except in very very specifically controlled situations, preferably with someone like a VFX supervisor who understands the full image path and can assess the impacts for the specific application.

Here is an example of a spot I shot recently that heavily used stabilization, but due to the client's request for it be a "stop motion esque" look they're all helping the aesthetic not hurting it. If the direction wasn't a stylized stop motion look I would have insisted on a gimbal.

 

[Jon D Kline]

You're also assuming that the stabilization need would be minimal. If your shake exceeds the difference between 4k and 6k, you'll need to add shake back in or guess what the shot should look like.

The MoVI can be a pain to rig up, but I have to say it's definitely a better solution than "stabilize it in post." There are plenty of consumer cameras that use DIS, but they don't do any of the advanced stuff on the fly, just a quick fix for horizontal and vertical positioning.

 

[Jacob Callaghan]

I do agree. I guess i was thinking in more of a minor stabilization sort of way. Using things that are simple and rugged like a sort of fig rig but still being able to get a smooth shot end result with post stabilizing. In the end not that great of an idea.

 

[ericyoung]

I do agree. I guess i was thinking in more of a minor stabilization sort of way. Using things that are simple and rugged like a sort of fig rig but still being able to get a smooth shot end result with post stabilizing. In the end not that great of an idea.

It's not a bad idea - it would work for certain specific types of shots. However, it's almost always better/easier/faster/cheaper to stabilise as much as possible in camera and then use a small amount in post to give a bit more smoothing as you've concluded.

 

[Gavin Greenwalt]

Yeah, don't get me wrong, I shoot a lot of plates out of airliner windows (saves money to just shoot the aerial plates I need without hiring a helicopter) and even OIS isn't nearly sufficient to handle the minor amount of turbulence found in most aesthetically pleasing clouds. I also don't have room even in a first class seat for a movi, nor the inclination to start building scary looking electronics at my seat . So for that I just crank down the shutter to like 10* and rely on Adobe's warp stabilizer which does a pretty decent job. But even then there are a lot of shots that just don't work. It's possible, it just turns every shot you do it on into a VFX shot which is slow, expensive and less reliable. If you can afford to pick and choose between unusable shots, great. If it's a narrative piece where you can't just drop every other shot because of egregiouis glitching then you