Can The Foundry "Freeze" Stills from Motion?

[Justin O'Neill]

It occurred to me in the shower (yes I think about HDRx in the shower) that Graeme and The Foundry may be able to use the two streams from HDRx to "freeze" still frames taken from a motion clip with a 1/48 shutter.

If they can change the blur (MNMB) why can't the get rid of it altogether? The shorter exposure should have all the clean edges for a still but would just be underexposed in the shadows. Just map in the properly exposed stream minus the motion blur and you are good to go. This wouldn't cover everything (like 1/2000s) but would always allow you to pull a clean still from your motion in normal situations.

We have found there are a surprising number of good still frames in 1/48s motion we shoot on the RED One but there plenty of instances where we can't find a frame sharp enough. If this is possible it would bring the DSMC concept to a new level.

Graeme, Simon? :smile5:

 

[AnthonyFlores]

Good question! Although I don't fully understand the technology, I'm interested in pulling the best possible stills from motion as well.

Anthony

 

[Gavin Greenwalt]

Adding motion blur is relatively easy. Removing it is extremely difficult. Adding motion blur destroys detail. Removing motion blur creates detail.

There are ways to reduce noise in an under exposed image using a blurred image. A few years ago Microsoft R&D presented a paper. But it's still extremely glitchy and produces pretty poor image quality. It also doesn't extend dynamic range below black in the underexposed image so you are still stuck with the same black point which might leave you with less than 4 stops of dynamic range with +8 HDRx™™.

It also assumes that all of the blur in the image is global from camera motion and the scene is static so if you want to remove the blur from someone running it wouldn't really work with currently demonstrated methods. I doubt optical flow could provide anything useful either. You would need some sort of 'coded shutter'.
http://www.cs.ust.hk/~quan/publications/yuan-deblur-siggraph07.pdf

If you just want to remove camera motion blur Microsoft (noticing a pattern? ) demonstrated a technique using Gyros (not sure if the epic/scarlet gyros are sufficiently accurate) http://research.microsoft.com/en-us/um/redmond/groups/ivm/imudeblurring/
But again... the results are pretty poor by professional imaging standards. You would be better off with a still from consumer point and shoot.


EDIT: Here is a link to a coded shutter paper http://web.media.mit.edu/~raskar/deblur/

 

[Peter Majtan]

It occurred to me in the shower (yes I think about HDRx™™ in the shower) that Graeme and The Foundry may be able to use the two streams from HDRx™™ to "freeze" still frames taken from a motion clip with a 1/48 shutter.

If they can change the blur (MNMB) why can't the get rid of it altogether? The shorter exposure should have all the clean edges for a still but would just be underexposed in the shadows. Just map in the properly exposed stream minus the motion blur and you are good to go. This wouldn't cover everything (like 1/2000s) but would always allow you to pull a clean still from your motion in normal situations.

We have found there are a surprising number of good still frames in 1/48s motion we shoot on the RED One but there plenty of instances where we can't find a frame sharp enough. If this is possible it would bring the DSMC concept to a new level.

Graeme, Simon? :smile5:

The Foundry's "DeBlur" from the FURNACE set of plugins for Nuke already does that... (and of course supports direct R3D)

http://thefoundry.s3.amazonaws.com/downloads/Furnace4.2v2_Nuke.pdf

:devil: Peter

 

[Jeff Kilgroe]

There's a good chance that Magic Motion and the X Track from HDRx could help to push this process further along. I can't wait to get my hands on it and see!

 

[andrewhake]

What really got me thinking with The Foundry being more involved in RED, is future camera modules...

Maybe a module that generates 3D camera tracking data on the fly that could be brought right into Nuke. Using the imagery the camera generates and actual gyroscope data from a module and maybe something like that would be possible.

 

[Cail Young]

Using the imagery the camera generates and actual gyroscope data from a module and maybe something like that would be possible.

There is a gyro built into the camera.

 

[Charles Angus]

The Foundry's "DeBlur" from the FURNACE set of plugins for Nuke already does that... (and of course supports direct R3D)

http://thefoundry.s3.amazonaws.com/downloads/Furnace4.2v2_Nuke.pdf

:devil: Peter

I've never had good results with this...

 

[Gavin Greenwalt]

I've never had good results with this...

I got good results once when I needed it. But it was only "Good" in that it was good enough to make some modifications and reblur. Certainly not good enough for a still photo or to stand up to scrutiny or replace a still photographer.

And DeBlur will only again work best with camera motion on a still scene. Don't ever expect to get a leg for instance to ever look right if it's too blurred from walking.

 

[Lauri Kettunen]

... to complement on what Gavin says, the principle is simple: motion blur is about loosing spatial details while capturing the image. Obviously, there is no way to recover. Correspondingly, any deblur algorithm is based on cultivated guessing and heuristics. There can be cases when such guessing may work out to some extent, but the unavoidable other side of the coin is, there are always far more cases when it doesn't.

The other way around, starting from frames that have spatial details, throwing them away is not a problem: If one has tomatoes, it's easy to make ketchup. But reproducing the tomatoes from ketchup is a quite a bit more challenging.

 

[Andy Jarosz]

I'll try this myself later and see how it works, but I'm thinking that if you generate a matte with the sharp version and apply it to the blurred version, you'd get a pretty cool effect.

 

[Subhadip Sen]

I'll try this myself later and see how it works, but I'm thinking that if you generate a matte with the sharp version and apply it to the blurred version, you'd get a pretty cool effect.

I have a feeling you would get a variant of Magic Motion. I have often wondered how this applies to shadows, though.