Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

Stills from Motion / Scarlet as 4k 3D Scanner

M

Mondo Ghulam

Well-known member
Joined
Jan 9, 2011
Messages
208
Reaction score
0
Points
16
Location
London
Hey,

When I first heard about stills from motion, I got very excited. The reason for my excitement was that I'd be able to do 3D capture more easily than I would be able to with a stills camera. I've been doing a few experiments when time permits and luckily today I got some nice weather to grab a section of a stone jetty in Edinburgh.

Image capture for mesh creation is one of those instances where you really want as much choice after the fact. Whilst you can do the same with a stills camera, it is a slower process and one that requires a little more thought ahead of time and while you're doing it. Essentially, having a free-running camera allows you to 'swab' your subject and make selections after the fact.

Of course, you need some software to convert the images to 3D meshes. I use Agisoft which is $179 and amazing for the price. You can also use Autodesk Photofly for free right now.

Using RCX Pro is great for this process, as you can handle the stream of images with selects saved to a project. This allows easy and accurate recall of all your selections after the fact. The benefit is that you can grade the image for mesh creation (you want contrast but not too many shadows) then later on, you can re-project the graded images to the mesh allowing you to match whatever grading you applied in RCX or whatever other software you happened to use.

This is the first time I've tried to post some images, I uploaded them to flickr - I hope it works.

Some info about the setup:

Scarlet-X with Canon EF 10-22mm lens. Shot at 4K, 6:1, 24fps, f14, iso 320, shutter speed was 192 - very fast to remove any blur.

The first run of images are grabs from Maya's viewport and are not full texture rez. I'll get some renders with a 4K texture as soon as I can.

http://www.flickr.com/photos/m_ghulam/6999647374/in/photostream
http://www.flickr.com/photos/m_ghulam/6999657664/in/photostream
http://www.flickr.com/photos/m_ghulam/7145744699/in/photostream
http://www.flickr.com/photos/m_ghulam/6999657672/in/photostream

Mondo
 
Mondo- Those meshes turned out great- can't wait to see them fully textured. How is the latest version of Photofly right now? I did 4 or 5 tests with the beta last year and never got results that were production-worthy...... For each test I experimented shooting using various techniques to minimize distortion and help the solver, and I even manually located the same points in the different shots.
 
Thanks Pax,

The mesh has come out really clean, that one is 400k polys and no holes. I'm going to kick off a 1.4million poly build once I've got a few renders out :)

I haven't used Photofly in a while, I got put off by the cloud / upload aspect of it. At the time everything I was doing was NDA'd up to the hilt, so it was non-starter.

Photoscan has come on leaps and bounds and all of it under the hood - workflow's the same. From what I can tell, the solver has definitely been improved. It's by far the simplest and most forgiving tool I've found. For the $ I can't recommend it highly enough. The texture construction is serviceable, for quick results like I did here. I'd recommend something like glyph matte painting toolkit for Maya, as you'll be rendering out 4k images from each camera, instead of a 4k square from Photoscan. Also you'd be able to pack the texture any way you choose in Maya (if needed).

It is SO easy to do this with the Scarlet. This literally took 290 frames to capture and about 1 minute to make the selects in RCX. Photoscan took around 45 minutes to build.

Mondo
 
Our company is using the Epic-X for facial capture mesh generation @ 120fps, with XYZRGB software for the initial mesh solve, and custom software for the rest. Works amazingly well.

I'm kind of surprised more people haven't figured out how useful the Epic is for 3D model generation of all kinds.
 
https://www.youtube.com/watch?v=Df9WhgibCQA

^^ Where things are going.

There's a similar solution here: http://matterport.com/
and an open source one here: http://pointclouds.org/news/kinectfusion-open-source.html

We already use "crappy" 3D mesh acquisition for rough and ready previs at Fohr.

The next step is to combine that with camera mapping once we have specific shots locked. The rough geometry looks like it should make a very high-quality camera mapping target to project onto, potentially saving a bunch of time there, too.
 
@Erich - that DTAM vid is remarkable. For me there are several possibilities with this stuff, the first to generate a 3D record of any location one shoots in, second is to create a project-specific library of assets that can be used in 3D for previz etc., lastly is to use the 3D data as an aid to any tracking being done in Syntheyes. Sounds like you're already quite far down this road already!

@Gavin - you can download it for free, it is super simple to use - 3-steps, cup of tea and you're done. I got into it for talent acquisition. With Photoscan and a 7D I could go from nothing, capture, process, optimise, rig and have a talent-derived mocap-ready asset in just under 6 hours. It just got a lot easier with the Scarlet and RCX Pro.

I still need to figure out how to get the camera vectors into Maya. Via the 3DS export, it comes into Max fine. The Maya 3DS importer doesn't seem to carry it through. Once I have that, it'll be n x 4K camera projections. Ooh la la!

More images coming soon.

Mondo
 
I received the "track,match,blend" dvd for blender ... which I found real solid http://www.blender3d.org/e-shop/product_info_n.php?products_id=147 .
On the blender front, you first do your camera solve, then you mesh map as needed to the surfaces. With the new VFX pipeline, it really opens up a lot of stuff for us Scarlet guys.
 
Mondo Ghulam said:
@Erich - that DTAM vid is remarkable. For me there are several possibilities with this stuff, the first to generate a 3D record of any location one shoots in, second is to create a project-specific library of assets that can be used in 3D for previz etc., lastly is to use the 3D data as an aid to any tracking being done in Syntheyes. Sounds like you're already quite far down this road already!

@Gavin - you can download it for free, it is super simple to use - 3-steps, cup of tea and you're done. I got into it for talent acquisition. With Photoscan and a 7D I could go from nothing, capture, process, optimise, rig and have a talent-derived mocap-ready asset in just under 6 hours. It just got a lot easier with the Scarlet and RCX Pro.

I still need to figure out how to get the camera vectors into Maya. Via the 3DS export, it comes into Max fine. The Maya 3DS importer doesn't seem to carry it through. Once I have that, it'll be n x 4K camera projections. Ooh la la!

More images coming soon.

Mondo
Click to expand...

If you have Max and Maya you could just import it into max and export the camera as an FBX.
 
@Gavin,

Max is the one 3D app I don't have :(

I was able to get it out as COLLADA, into Softimage and into Maya with cameras.

I noticed that there have been a few requests for an FBX exporter on the Agisoft forum, that would simplify things tremendously.

I'm hoping to get some more images and videos up later today - 18x 4k camera projected images from Scarlet...

Mondo
 
How do you feel that the Scarlet's very apparent rolling shutter is affecting the results? I'm sure the data are compromised (errors and averaging introduced) unless you are moving very slowly in your walkaround.

I imagine EPIC with it's faster shutter read rates would be a superior tool for this.

How quickly are you moving the camera for these scans?
 
Hey Eric,

The stills are snapped out of a 11 second clip. I was on a beach and mis-stepped a couple of times but I couldn't discern any rolling shutter in the clip. If there was, I'd just have used an adjacent image - that's the beauty of stills from motion. I can upload the source clip at some point if you'd find that useful.

Overall though, I don't feel rolling shutter was or would be an issue for this type of work - the subject needs to be static for Photoscan. You can move at a reasonable speed and still get very useable results, certainly faster than you would firing off stills.

On a side note, I was under the impression that the rolling shutter characteristics between the two cameras were the same:

http://www.reduser.net/forum/showth...ed-Epic-worthy&p=866175&viewfull=1#post866175

Mondo
 
You must log in or register to reply here.
Back
Top