Rolling shutter test with #232

[Thom Steinhoff]

Another "classic" solution when shooting 100fps (or 96) for the example with the rescue cars and their flashlight:
In 100P you get 4 frames, now combine these 4 frames in post - there are several packages which offer this functionality built in (discreet combustion).

This sounds like a very interesting "low tech" solution. I wonder how it would look? It would be interesting to see various takes on it: 48 combined down to 24, 72 combined down to 24 or 96.

Especially if the software knew it was trying to fix skew and strobing and could take that into account while doing the combine: Rather than just some sort of averaging, knowing that this is 4 consecutive frames over a 1/24th time period, now lets use these 4 shots, knowing there may be skew, to correct for it and give a nice, motion blurred image of what must be there.

 

[laguun]

This sounds like a very interesting "low tech" solution. I wonder how it would look? It would be interesting to see various takes on it: 48 combined down to 24, 72 combined down to 24 or 96.

It is also used for a classic "mishap" - people shooting under 59.xx light with 50 or vice versa.

In 2005 we fixed ~80% of a doc which was shot at the US/Mexican border with 50i, lots of hidden cam shots, lots of available light only (was a doc regarding the "disappearing" girls there). DP had only a progessive Viewfinder, didn´t see that his shots were flickering all the time until post.

Especially if the software knew it was trying to fix skew and strobing and could take that into account while doing the combine: Rather than just some sort of averaging, knowing that this is 4 consecutive frames over a 1/24th time period, now lets use these 4 shots, knowing there may be skew, to correct for it and give a nice, motion blurred image of what must be there.

Easy one, howewer computing intense.
Example with the policecars:
Take 4 layers with framecombine, shift start-in every by 1 frame (master) equals 0.25 frames (target). Add slight luma key on any of the top 3 layers.

Now you should get a nice moblur on the highlights and a pretty sharp source image.

To enhance the dynamic impression of the light: change the transfer mode to add or multiply. I am mentioning discreet combustion as i am 100% sure it offers this function AND is available for a 15 day trial online AND does come in OSX and Windows flavour, in case someone wants to experiment.

 

[Thom Steinhoff]

Laguun,

Sounds very cool. I'd love to see a test on this. Sounds like this technique would be the best way to go.

Of course it is a good workaround where a solution would be better, but to me it sounds like most solutions will require hardware advancements that aren't likely to happen this round.

At least this may be a viable fix for trouble shots.

So, who's going to shoot some tests?

Whip pan, police car & strobes at 24 fps in 2K to compare to 48fps, 72, & 96 combined in combustion?

 

[laguun]

Laguun,

Sounds very cool. I'd love to see a test on this. Sounds like this technique would be the best way to go.
At least this may be a viable fix for trouble shots.

won´t work for motiontracking and greenscreen however

 

[Thom Steinhoff]

won´t work for motiontracking and greenscreen however

Never quoted myself before, but I posted this in the other "How to fix skew" thread in the main forum. What do you think of this concept?

This talk of tracking gave me another idea.

It would seem to me that some smart vfx developer could take the fact that shot X can't track properly, place simple "This part of the frame is fixed to this part of the frame" markers (such as on the edge of a building as it pans by), and use those markers to do "Skew compensation" on the frames.

Yes it is a post process, and yes it would be great if the camera doesn't have skew in the first place--but at least it would be easy software fix and once fixed--easy to track.

Sure it doesn't do anything for stobes but at least skew would become post fix.

 

[JarredKrakow]

line scan order

line scan order

Is the line scan order hardwired into the cmos or is it possibly software? I'm 99% sure it's hardwired, but I had to ask.

 

[laguun]

Never quoted myself before, but I posted this in the other "How to fix skew" thread in the main forum. What do you think of this concept?
This talk of tracking gave me another idea.
It would seem to me that some smart vfx developer could take the fact that shot X can't track properly, place simple "This part of the frame is fixed to this part of the frame" markers (such as on the edge of a building as it pans by), and use those markers to do "Skew compensation" on the frames.

Sure it doesn't do anything for stobes but at least skew would become post fix.

Fully possible - and already on the market since over one decade.
If we stay with discreet products, you would use four corner tracking (a ka corner pin) to get this result.

However, there is one mayor issue with this approach.
usually, objects in the image tend to disappear pretty fast out of the image when panning fast.

Therefore only pretty constant pans would really be feasible w/o much manual work. Also, in perspective different objects will have different paralax speeds, closer objects will show a stronger rolling effect than remote ones, which also reduces the useful applications of a tracked skew.

However, fast pans usually can be softened with directional motion blur (@ discreet combustion, its even called pan blur) in order to reduce the strobing impression which 24P gives when panning fast.

When mastering especially VisualFX etravaganzas one often has to use several panblurs on different layers, in order to trick the audience when combining motionblurred real footage and razorsharp cgi.

Furthermore, there have always been maximum panspeeds for 24p, many old-school guys can be pretty precise with them, and cmos rolling shutter should be pretty uncomplicated if respecting these.

Modern narrative styles however often use shaky/trashcam camera moves especially to heighten action and do not longer follow the classical guidelines.

 

[Paris Remillard]

Is the line scan order hardwired into the cmos or is it possibly software? I'm 99% sure it's hardwired, but I had to ask.

I'd like to know as well. But even if it's hardwired, it seems that the sensor would just need to be either cut (from the wafer I'd assume?) 90 degrees from how it's currently cut, or rewired. If we're talking about reorienting the scan direction. But I'm completely ignorant on these issues, so...

 

[Jan Reiff]

faced with all this stuff of rollin shutter, i am so very excited about the forthcoming movie of soderbergh and how it will look.

 

[Gregory Middleton]

Jim,
Thank you very much for the work you put in on that test.
I shot a 35mm comparison test and noticed the skew/rolling shutter issue. It was a real surprise.
Its a bit of a terminal problem for me on a show with many tracking shots.

 

[Paris Remillard]

Jim,
Thank you very much for the work you put in on that test.
I shot a 35mm comparison test and noticed the skew/rolling shutter issue. It was a real surprise.
Its a bit of a terminal problem for me on a show with many tracking shots.

Gregory, would you mind telling what 35mm camera was used and if there was any noticable skew in that footage? Also, what is the shutter mechanism on that camera. Namely, the position of the mirror shutter and how it swipes the frame. If it's underneath, it will swipe from side to side. If it's on the side, it will swipe from top to bottom. I ask because the the direction of the shutter swipe, whether the mechanical film mirror/shutter or the rolling CMOS shutter, will affect what artifacts pop up, either skew or expansion/contraction.

You may want to try mounting your Red on it's side and doing the same test, then crop a 2K frame out of the middle. It shouldn't have any skew, only expansion/contraction, which will probably be less noticeable on casual viewing.

To me, the worst is when both skew and expansion/contraction happen at the same time and the image gets squishy and jello-y. But my short term plan for major skew on horizontal motion is mounting the camera sideways and cropping out the center. We'll see how it works when my camera finally shows up this week. Good Luck.

 

[indivfx]

we have done several syntheyes 3d motion tracking tests with red footage off this board.... no problems at all extracting 3d data. Easier to get a stable sollution than any footage i've ever tracked before.

What tracker was used Obin? Is it Syntheyes or Boujou?

Hari

 

[Joe G.]

Some obvious workarounds that weren't mentioned:

Police cars - You want to shoot your actors, not strobe lights, no? Leave the strobes off and add the lighting effects in post. If the cars aren't creating odd artifacts, you should be able to mimic the effects to an acceptable degree. (Maybe? Yes, no?) This depends on what's actually in frame, and leaving the police cars out (save money, eliminate the actual cop cars, fire trucks and ambulances entirely).

Similarly, Robert Rodriguez shot a lot of Once Upon a Time in Mexico with guns that didn't fire at all. The muzzle flashes were added in post, as were a lot of the bullet holes, in buildings he wasn't allowed to actually shoot up or damage.

If you must shoot strobe lights because of a big moment with lots of emergency vehicles -- why not get a 16mm grainy shot for that sequence? (Perhaps it's a cliche, but if it contrasts with the rest of the film, the director might like the idea). Renting another camera for a day would seem an obvious fix.

I'm personally much more worried about the skewing, for which there appears to be no fix at all, except to keep motion from crossing the screen too fast. I would like to add sfx in layers, and am worried if some of the images are skewed and some aren't, if this will be noticeable or not? Could be a dealbreaker.

When's that "global shutter" going to be a reality?