Welcome to our community

Be a part of something great, join today!

Register Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

Weapon gyro data for stabilization?

Christoffer Glans

Christoffer Glans

Well-known member
Joined
Jun 11, 2007
Messages
4,993
Reaction score
36
Points
48
Age
42
Location
Stockholm, Sweden
I read that the gyro data has been increased to capture once every 24 or 25 frame per second. Is there any way to export this data or transfer this into a virtual camera or stabilization tool in applications like Resolve or After Effects? I've been keeping an eye on the SteadXP box which delivers perfect post stabilization by recording gyro-data when shooting and later stabilize the footage based on that data. Instead of tracking the camera in post, which is impossible in cases like heavy snow or other movements, having the actual camera tracked while filming is a hugely underrated feature... if only we could pull that data out of the Weapon?
 
Antony Newman said:
I think RED advised the introduced a per frame Gyro reading.

Not sure what the max fps this works up to.

AJ
Click to expand...

Yes, but how do we use it, is there any way to extract and import that data into a virtual camera/stabilization?
 
It's been said that the SDK for this is already in place for quite some time, only that no software is taking advantage over this.

Reduser Mikael Lubtchansky said he was working on some kind of extractor. Maybe you can contact him for further info?

Mikael Lubtchansky said:
Have been extracting and playing with some of this data since it has been available via REDLine...
Not ready yet but hopefully soon :-)

1549216_824376241007097_7365770454065103580_n.jpg
Click to expand...
 
can not be too difficult to extract. Where is the numbers stored? Do you see them if you jot the rmd? or is it stored in the header of each frame? I hope its embeded in each frame that would make things more convinient when doing r3d trims etc.

Any way, its just to write a script that extract the numbers for rotation and preferably translate them to degrees.

Then its simple enough to import the strings as raw onto an axis y/x/z rotation and link your 3d camera to it and set the camera to slightly less fov. Then you negate the curve and scale them until each axis is silent.

Then you know the math is right. Normally you need to rectify your lens first. But from there you can use normal stabilize features like like remove gitter over frames etc.

but its not an exact science the gyros are not 100% correct and also to get it right you would need to know where the center of rotation for the gyros are and then calculate your lens nodal offset. As the pan tilt roll of the physical camera is not based around the nodal point of the lens so that means a camera pan tilt roll is not exactly what happens to the picture which get an additional paralax change from those movements.

And you can only fight rotation this way all other moments is a different beast to fight and normally demand a pixel based 3d track and the image to be projected onto scene matched surfaces.
If we are not too occupied with other stuff I got a guy at my office that i think could easily turn this into a nuke plugin. For sombody that program and knows nuke scripting i think it can not be that difficult.

Still I think when do e we will realize that a good old image based 3d track will be a much more solid ground.
 
Björn Benckert said:
can not be too difficult to extract. Where is the numbers stored? Do you see them if you jot the rmd? or is it stored in the header of each frame? I hope its embeded in each frame that would make things more convinient when doing r3d trims etc.

Any way, its just to write a script that extract the numbers for rotation and preferably translate them to degrees.

Then its simple enough to import the strings as raw onto an axis y/x/z rotation and link your 3d camera to it and set the camera to slightly less fov. Then you negate the curve and scale them until each axis is silent.

Then you know the math is right. Normally you need to rectify your lens first. But from there you can use normal stabilize features like like remove gitter over frames etc.

but its not an exact science the gyros are not 100% correct and also to get it right you would need to know where the center of rotation for the gyros are and then calculate your lens nodal offset. As the pan tilt roll of the physical camera is not based around the nodal point of the lens so that means a camera pan tilt roll is not exactly what happens to the picture which get an additional paralax change from those movements.

And you can only fight rotation this way all other moments is a different beast to fight and normally demand a pixel based 3d track and the image to be projected onto scene matched surfaces.
If we are not too occupied with other stuff I got a guy at my office that i think could easily turn this into a nuke plugin. For sombody that program and knows nuke scripting i think it can not be that difficult.

Still I think when do e we will realize that a good old image based 3d track will be a much more solid ground.
Click to expand...

Have you looked at what SteadXP are doing? It's the same, only that the tracking in this case happens in the camera directly instead of an external box. But you make programming sound easy hahaha :tongue_smilie:
 
Christoffer Glans said:
Have you looked at what SteadXP are doing? It's the same, only that the tracking in this case happens in the camera directly instead of an external box. But you make programming sound easy hahaha :tongue_smilie:
Click to expand...

Yes there is a few options, some also use the Xsens sensors. They stream over Blue tooth so you get a live feed for the data and track real time, and they do no cost much and comes with a SDK etc. We used to have such like 10 years back.

But those kind of sensors like the steadyXP and Xsense will only sell until everyone understand that they can get pretty much the same or better by gaffer taping an Iphone to their camera and run something like the "Gyro Recorder" app that cost about 3 USD and then import the XYZ rotations into flame, nuke, fusion, maya, max or what ever program you want. And the iphone gyro is not bad, on the contrary it´s better than most.

So tape your iphone to your camera. Hit record on the iphone hit record on the camera then hit the camera hard on it´s head with your hand so you get a good bump in picture and gyro data to cue and and then film away then end of take hit the camera hard again. The two que marks makes it easy enough to stretch the string of keyframes to the right length.
If you are on a tripod that data should be more than enough to put CG into the scene. If you are running and gunning it will for sure help to keep horizon level if you only extract the Z data import it in your compositor , negate it and Y scale until shit is not rolling any more. Then most likely the same scale is applyable for both pan and tilt. But those are parallax changing axises so you can counter steer them but only pan tilt not the up/down forward / backward or side to side move... Simply only the same axises a gimbal can handle.
 
Björn Benckert said:
Yes there is a few options, some also use the Xsens sensors. They stream over Blue tooth so you get a live feed for the data and track real time, and they do no cost much and comes with a SDK etc. We used to have such like 10 years back.

But those kind of sensors like the steadyXP and Xsense will only sell until everyone understand that they can get pretty much the same or better by gaffer taping an Iphone to their camera and run something like the "Gyro Recorder" app that cost about 3 USD and then import the XYZ rotations into flame, nuke, fusion, maya, max or what ever program you want. And the iphone gyro is not bad, on the contrary it´s better than most.

So tape your iphone to your camera. Hit record on the iphone hit record on the camera then hit the camera hard on it´s head with your hand so you get a good bump in picture and gyro data to cue and and then film away then end of take hit the camera hard again. The two que marks makes it easy enough to stretch the string of keyframes to the right length.
If you are on a tripod that data should be more than enough to put CG into the scene. If you are running and gunning it will for sure help to keep horizon level if you only extract the Z data import it in your compositor , negate it and Y scale until shit is not rolling any more. Then most likely the same scale is applyable for both pan and tilt. But those are parallax changing axises so you can counter steer them but only pan tilt not the up/down forward / backward or side to side move... Simply only the same axises a gimbal can handle.
Click to expand...

I would actually pay you to do a tutorial of this and then use the data in After Effects or similar software :D
 
If nothing pops on this front at NAB, I am willing to pitch in on a dev effort to make this kind of stabilization ready for production. No offense to the RedTeam - they've accomplished a great deal in the last 2 years, but I expected this capability would be part of RCX-P by now...

Cheers - #19
 
Hmm, so I got +1 money x3 if I pull this stunt off. :)

The shit part of all this is the fact that I hardly even know how to open after effects. But I have some time tomorrow and now when you guys put the pressure on maybe it´s a good time to learn that app . :)

I mentioned a bad gyro app before this is better: https://itunes.apple.com/us/app/sensor-kinetics-pro/id623633248?mt=8

it cost 3 dollar and if you down load it go into the gyro and then hit record and move the phone around then stop and email your self the file.... then you half way there.

Bu there is money in this now so I can not tell you more :)

Will show you tomorrow :)
 
Blair S. Paulsen said:
No offense to the RedTeam - they've accomplished a great deal in the last 2 years, but I expected this capability would be part of RCX-P by now...
Click to expand...

RCX team did their job, gyro metadata export is part of redline and has been for long time, the camera team on the other hand....

Redline.exe --printmeta 5 -i YourClip.R3d > YourClip.CSV

You end up with a txt file with:

FrameNo,Timecode,Aperture,Focus Distance,Focal Length,Acceleration X,Acceleration Y,Acceleration Z,Rotation X,Rotation Y,Rotation Z

You can then bring that into AE, Maya, massage it with excel/google docs, etc.

The problem is the data is pretty unusable from the camera gyro - it is per frame now, never tested max frame rate, but maybe its interpolated or something because it has lots of jitter, random large variations, etc.
You can spend time cleaning it up but it never seems worth it.
 
Christopher*Seguine said:
RCX team did their job, gyro metadata export is part of redline and has been for long time, the camera team on the other hand....

Redline.exe --printmeta 5 -i YourClip.R3d > YourClip.CSV

You end up with a txt file with:

FrameNo,Timecode,Aperture,Focus Distance,Focal Length,Acceleration X,Acceleration Y,Acceleration Z,Rotation X,Rotation Y,Rotation Z

You can then bring that into AE, Maya, massage it with excel/google docs, etc.

The problem is the data is pretty unusable from the camera gyro - it is per frame now, never tested max frame rate, but maybe its interpolated or something because it has lots of jitter, random large variations, etc.
You can spend time cleaning it up but it never seems worth it.
Click to expand...

Where do we find redline? I only find a linux beta that does not run well on my mac when starting it.
 
Björn Benckert said:
Where do we find redline? I only find a linux beta that does not run well on my mac when starting it.
Click to expand...

REDCINE-X -> show package content
REDLine is in contents / MacOS

But as Christopher noted the gyro data itself does not seem to be accurate enough in-camera yet to do any real stabilisation ...
 
Mikael Lubtchansky said:
REDCINE-X -> show package content
REDLine is in contents / MacOS

But as Christopher noted the gyro data itself does not seem to be accurate enough in-camera yet to do any real stabilisation ...
Click to expand...

That's a bummer. I know that as Björn said, we could use an Iphone for recording, but the main thing is synchronization. Having it as metadata on the actual filmed footage is powerful in that if the recorded data was accurate enough, it would almost be like a one-switch or slider to stabilize the footage in post. What you CAN do is not the same as the cost of time with doing complex calculations for stabilizing the footage. That's what I like about the SteadXP, it records timecode and syncs to the footage. But if Red could have accurate recording to the R3D directly, that would be huge for any kind of stabilization work. Maybe we could get some external recording device that syncs with cable or wifi to the camera. Maybe something through fool control? :thumbsup:
 
You must log in or register to reply here.
Back
Top