STEADXP on RED.

[Björn Benckert]

I have given up the hope of the built in gyro on dsmc1 and maby even dsmc2. But i have hi hopes for hydrogen will have seriously good gyro onboard for this.

Your test looks very promesing Björn. How fast is it to stabilize in post? Warp stabilizer on a 8k clip is not so snappy. Also the few times i like the warp stabilizer is when converting a shoulderrig shot to static. Funny thing is you always have to turn of the warping in the warp stabilizer cause it always looks ugly.

The actual render time is like rendering a quicktime. I assume much faster then the warp stabilizer that actually do quite intense math to come up with the result. This thing you can play trough if your computer can play the file. But it wants the source and it looks for sync with the data file and there is for sure more fiddle than dropping the FX icon onto a premiere timeline clip. But then yes you will have much better result and you can do it for any length clips where you spinn around with the camera in circles, look at the floor, look up spin around etc. Warp stabilizer usually don't approve such behavior and care very much about what goes on in front of camera. If you use warp stabilizer on a shot with a big car driving by close to camera it simply fail, this does not.

 

[Scot Yount]

Looks cool...

 

[paulcurtis]

Looks interesting.

I'm also divided on post stablisation, it is very much shot dependent. I found a lot of the work in Birdman utterly distracting from that perspective for example. And in our own work there have been plenty of times where i've done it and then removed it.

This is interesting, it sounds like it will compensate for skew but does that include vertical too - will it stretch and scale a vertical move?

I'd been using Mercalli as an obscure stablisation product because it does make an attempt to rewarp jello in a way that i've not seen from any other product. Sometimes, it can work wonders and when processing you see it mapping the distortions of the sensor. But there are plenty of cases where it just can't do it. I've also done the whole thing by hand, in Nuke, Mocha, PFTrack and pretty much everything.

I didn't know there was gyro in the Red files. Suddenly i'm curious as to how accessible this is. I wonder if Nuke can get the metadata... Because beyond stablisation having some gyro data for 3D tracking would be super useful too!

One issue i've found is that going in if you know you're going to have to stabilise then the temptation is to run a higher shutter speed, things work better and you don't get blur buzzes in the middle of a stable move. But adding motion blur back can also be quite shot dependent too - crossing objects and so on. What would be interesting in this case is that can the gyro data be used as motion vectors to do a blur pass?

cheers
Paul

 

[Björn Benckert]

Looks interesting.

I'm also divided on post stablisation, it is very much shot dependent. I found a lot of the work in Birdman utterly distracting from that perspective for example. And in our own work there have been plenty of times where i've done it and then removed it.

This is interesting, it sounds like it will compensate for skew but does that include vertical too - will it stretch and scale a vertical move?

I'd been using Mercalli as an obscure stablisation product because it does make an attempt to rewarp jello in a way that i've not seen from any other product. Sometimes, it can work wonders and when processing you see it mapping the distortions of the sensor. But there are plenty of cases where it just can't do it. I've also done the whole thing by hand, in Nuke, Mocha, PFTrack and pretty much everything.

I didn't know there was gyro in the Red files. Suddenly i'm curious as to how accessible this is. I wonder if Nuke can get the metadata... Because beyond stablisation having some gyro data for 3D tracking would be super useful too!

One issue i've found is that going in if you know you're going to have to stabilise then the temptation is to run a higher shutter speed, things work better and you don't get blur buzzes in the middle of a stable move. But adding motion blur back can also be quite shot dependent too - crossing objects and so on. What would be interesting in this case is that can the gyro data be used as motion vectors to do a blur pass?

cheers
Paul

The red gyro data is allready in the meta data per frame so not only in the camera. I guess hollywood is allready using it.

The "post stabilisation" of Bridman is a bit wonky not so much because it is stabilised more because every take is "stitched" into the next and as they use steady cam naturally the seam between two takes does not match prerfectly so they need to warp the prespective to get them together.

And yes, steadXP takes vertical scew into account it use the time it takes per pixel row to be written into account. Having that number as a given makes it possible. Estimating that number is difficult for a software so it´s kind of something that has to come from red. or you need to be able to make very precise tilts and pans to figure it out. But its close to 15ms for the full height of monstero in 25fps.


The trick that can be done about motion blur is a bit difficult. But for example you can shoot 100fps 360 shutter. Then you stabilize that. After stabilisation you blend frame 1 and 2 together and ditch frame 3 and 4. That way you get less than half the trouble wth the motion blur. If you got a phantom 4k or such you can do the same but in 1000frps and then you can mix over frame with something like a sinus curve. So you actually get a global fading shutter. But yes that would mean shooting huge amount of data toss half of it away and then process the other half quite heavily. But in theory that would give the best motion blur possible out of a cmos sensor.

But if you know what is image change due to pan tilt roll you can create better optical flow motion blur. Not at the level of the real deal but should be possible to make things way better than just using optical analysis.

 

[paulcurtis]

The red gyro data is allready in the meta data per frame so not only in the camera. I guess hollywood is allready using it.

Any idea where i can see the data. In RedCine X i haven't found any gyro data and i'm pretty sure not through Nuke either. I would love to find this...

The "post stabilisation" of Bridman is a bit wonky not so much because it is stabilised more because every take is "stitched" into the next and as they use steady cam naturally the seam between two takes does not match prerfectly so they need to warp the prespective to get them together.

For me it was blurred frames, you know when a lot of movement was stabilised but the frame was smoothed, so everything futz's for a single frame. They didn't rebuild these frames (budget probably)

And yes, steadXP takes vertical scew into account it use the time it takes per pixel row to be written into account. Having that number as a given makes it possible. Estimating that number is difficult for a software so it´s kind of something that has to come from red. or you need to be able to make very precise tilts and pans to figure it out. But its close to 15ms for the full height of monstero in 25fps.

The trick that can be done about motion blur is a bit difficult. But for example you can shoot 100fps 360 shutter. Then you stabilize that. After stabilisation you blend frame 1 and 2 together and ditch frame 3 and 4. That way you get less than half the trouble wth the motion blur. If you got a phantom 4k or such you can do the same but in 1000frps and then you can mix over frame with something like a sinus curve. So you actually get a global fading shutter. But yes that would mean shooting huge amount of data toss half of it away and then process the other half quite heavily. But in theory that would give the best motion blur possible out of a cmos sensor.

Would 100fps at 360 be sharp enough? Nice approach. In the past it's been a case of shooting like 1/200th and then optical flow and blurring based on that, but doesn't always work.

But if you know what is image change due to pan tilt roll you can create better optical flow motion blur. Not at the level of the real deal but should be possible to make things way better than just using optical analysis.

Yes, that was my thinking. So i'm keen to find the gyro data in the R3Ds!

cheers
Paul

 

[Terry VerHaar]

Have to admit, I find this very intriguing. Would love to see this implemented in camera but, in the meantime, an inexpensive external box is interesting.

 

[Ryan Sauve]

I also find this really fascinating! Is REDs gyro metadata available for someone to create a stabilizing software plugin for something like Premiere? If I can unload a few of the accessories I'm trying to sell, I'm going to buy me one of these and share all my tests. I might be able to use this on an upcoming project in a few months. Would be rad if I could pull it off.

 

[paulcurtis]

The only reference i could find online was saying that the gyro data isn't saved to the R3D (begs the question WTF is it for) but that was 2014. Then a bunch of posts more recently talking about it but with no real evidence.

There was a thread saying that per frame metadata was opened up and some examples from focus.

If the gyro data was in the metadata then we'd have so many more options...

cheers
Paul

 

[Ryan Sauve]

Just received mine a week ago and I should be able to start testing it tomorrow. But yeah, no batteries included, nor an audio cable, or cold shoe mount. I bought some GoPro 3 batteries online and the retailer (SportChek) sent me GoPro 4 batteries. Fortunately, I discovered that the Smatree 3rd party batteries I got that I ordered for the charger work with the SteadXP+ so I can get by with cheap 3rd party batteries, woot.

On tomorrow's agenda is to calibrate the system then perform a quick test shoot of time permits. I'll share my results. I'll be shooting 8K.

 

[paulcurtis]

Did we ever decide whether it deals with jello inside the image?

I had some problem footage that was shot on a long lens (not Red) with in camera stabilisation set to the wrong focal length. Can you imagine the microjitters? Nothing worked except for Mercalli. Literally it is the only thing that would map the whole sensor and correct the best it could at that level of jitter. Adobe results were hilarious by comparison.

Now i'm pretty sure this is just going to give broad movement and gyro - not down to the degree of sensor vibration or hand microjitters, am i wrong?

cheers
Paul

 

[Björn Benckert]

Did we ever decide whether it deals with jello inside the image?

I had some problem footage that was shot on a long lens (not Red) with in camera stabilisation set to the wrong focal length. Can you imagine the microjitters? Nothing worked except for Mercalli. Literally it is the only thing that would map the whole sensor and correct the best it could at that level of jitter. Adobe results were hilarious by comparison.

Now i'm pretty sure this is just going to give broad movement and gyro - not down to the degree of sensor vibration or hand microjitters, am i wrong?

cheers
Paul

You are wrong.

It calculates shakes per pixel line and redraw time for the camera.

So not just a per frame thing.

 

[paulcurtis]

You are wrong.

It calculates shakes per pixel line and redraw time for the camera.

So not just a per frame thing.

Interesting.

In the use case i gave though, if the device was mounted on the camera then it wouldn't be able to detect the sensor stabilisation in the camera to compensate? Literally a 135mm lens was being stabilised in the A7sII as a 35mm lens, so each frame jittered internally due to the sensor moving, not the camera.

It's a case of understanding to what degree of sensitivity it's running at. But that is interesting... Must take another look

cheers
Paul

 

[Björn Benckert]

Interesting.

In the use case i gave though, if the device was mounted on the camera then it wouldn't be able to detect the sensor stabilisation in the camera to compensate? Literally a 135mm lens was being stabilised in the A7sII as a 35mm lens, so each frame jittered internally due to the sensor moving, not the camera.

It's a case of understanding to what degree of sensitivity it's running at. But that is interesting... Must take another look

cheers
Paul

Not sure I understand, this thing can not correct if there is a in lens stabilizer used. It only calculate from how the camera body rotates and is able to derotate it. But on a sub frame level.

 

[Ryan Sauve]

Not sure I understand, this thing can not correct if there is a in lens stabilizer used. It only calculate from how the camera body rotates and is able to derotate it. But on a sub frame level.

Hey Bjorn, what calibration settings did you use? It only shows me "RED", "Epic", and "5k Ratio 2" for the drop down options. Trying to calibrate but getting "not enough chess boards were detected" when I've followed all the instructions.