SONY 3D.

[Neil B Smith]

Try putting this 3D setup on your shoulder. This is definitely NOT the future for 3D. Single image file storing 3D info (i.e. RGB-3D) using a single light-weight camera is. This type of 3D setup is dead before its got going. Think again anyone whoever thinks you need 2 cameras to shoot 3d. New technology is coming.

I think 3D can be stored in to a single file. Instead of an alpha channel replace it with a 3D depth channel. I use 3ds max to create depth maps all the time then use them in After Effects to create the DOF I want, easier and faster. RED can you be the first to create such a file type??

 

[Neil B Smith]

UPDATE on 3D Depth Camera.

UPDATE on 3D Depth Camera.

3D Depth cameras are already available. Using this technology you can create a true 3d image. This is the future of 3D filming, not 2 x 2D cameras.

http://www.mesa-imaging.ch

http://www.hizook.com/blog/2010/03/...-ranging-cameras-or-rgb-d-cameras-emerge-2010

It seems that this type of camera is mainly being used for Games generation, but expect it to leap over to film and television production in the coming years.

 

[Ketch Rossi]

Yeah ,

but don't forget that Sony also has Epic Sized camera Boxes that are currently been used with the TS-5 3Ality Rig, and those are tiny, and the Mini Zooms used in this rigs are also really tiny.

Sony does have many new cameras testing in prototype like Epic/Scarlett size, just not the same specs , but they don't really care, their market Nit-ch is Live S3D BroadCast.

 

[Neil B Smith]

Everything that is tech eventually goes tiny. Take mobile phones that needed a car battery to power them. The same will be for broadcast & film cameras. Also, as we are entering (or re-entering) the world of 3D filming the tech for 3d will go small, eventually you will have a camera with a single lens shooting 3D in the palm of your hand. Possibly at 4K or more resolution. 3D depth imaging seems the way forward in to this 3D future, not 2 separately filmed images. I don't know how far in to the future this will be, but I can see it happening.

 

[Gavin Greenwalt]

If you have 3ds max and post DOF then you know the horrendous artifacting that results from trying to use a depth channel to apply depth of field even with a *perfect* depth map on a single layer.

Most 3D depth sensors today are about 320x240 pixels and extremely artifacty. Even if we improve the resolution and reduce the artifacts they won't really be reliable until they can output a perfect 3D scene. If they can output a perfect 3D scene then you're effectively using performance capture anyway.

The future future is performance capture and scene scanning and then actually rendering two eyes. There are just too many artifacts and limitations to using something like even Lidar.

 

[Bruce Allen]

This is not a good representation of SONY 3D.

Pace Fusion is a better example. Eg Steadicam. Eg Avatar.

Bruce Allen
www.boacinema.com

 

[Alex G. Cohn]

How do you plan on displaying your 3D Depth images?

Short of building some sort of holographic display, it seems like the images would have to rendered out to double-image Stereoscopic 3D in order to be watched.

In which case, you might as well record Stereoscopic.

Plus, your method can't create an true 3D image, because the camera would only be capturing a single perspective.

The 3D effect, by its very nature, is created by combining two different perspectives of an object.

 

[Mark Collins]

SI-3D rig? Epic 3D rig?

The Sony camera's are large for sure, but you don't necessarily need camera's that large. I remember seeing some set photo's of Avatar where Cameron was going shoulder mount on the Pace system easily enough. Granted it's huge, but as long as the balance is there you're fine.

There's also system's such as the steadicam and the steadi-segway to use beamsplitter rigs with.

 

[Dan Hudgins]

Compression and 2D vs. 3D

Compression and 2D vs. 3D

People have two eyes.

The only way to get good Stereo 3D is with two cameras and to record at high resolution uncompressed.

The only way to view good Stereo 3D is using to high bandwidth uncompressed channels to two high resolution projectors.

The only way to shoot good stereo 3D is with cameras aligined like your eyes using a single focal length that matches where you are going to sit in the theater relative to the screen size.

You need MORE bandwidth and detail in the stereo pair for 3D than for 2D since your brain uses very small differences between the two images to derive the depth sensation, what compression looks "lossless" in a 2D image may not BE lossless in a stereo-pair.

As you degrade the images you get increasing "cardboarding" of the images, to the point where the audience does not know what "real" 3D looks like because all they have seen are computer graphics with soft 2D images mapped over them and called "3D".

By the way, its the out of focus parts of the image that give the sensation of 3D, not the part that makes up the point of convergence of the eyes, so to get that in a stereo pair you need accommodation depth clue in the display. Being out of focus does not mean devoid of information, and compression systems can distort the out of focus information.

==

If you have heard 6 discreat track sound from 70mm mag stripe, vs. sound systems that use difference recording methods you can tell one from the other as the heavy use of "center channel" in the non-discreat playback systems removes the "Binaural" effect for the most part, at least in the way most films I have heard mixed for such release.

Why don't you promote making ALL recordings with one microphone ONLY and then produce so called "Stereophonic" records from that mono recording using a spring reverb on the left channel and some high-pass and low-pass on the two playback channels like some Cheapskate record producers have in the past?

Afterall if ONE camera is good enough for Stereo-3D, then ONE microphone should be good enough for Binaural Stereo audio recordings, if you have "really good software" to make a synthetic result...

Afterall the public are morons and will pay for anything if you say its what they want often enough...

==

The issue comes up as to if "photography" or "cinematography" will be any part in making "Movies" in the future. Will all "Movies" be made using motion catpture or be totally Synthetic. With so much computer graphics being called "grading" what part of what the audance sees was visable at the time the actors gave the performance, and how much was "added later".

 

[Neil B Smith]

If the post production software knows Depth then you can begin to wiggle the (virtual) camera around in post. Instead of editing 2 separate images you would be editing with a single 3D image. It would then be up to the software on output to either supply a single 2D version (not 3D), or 2 separate images for the left & right eyes (3D as you all know). When rendering out the images for 3D (2 separate images) you can set the distance between each virtual camera, their angles, etc.., just as you would in the field. The beauty would be you can always go back to the edit and adjust the settings to get things just right. Setting up two cameras to be perfectly aligned for a 3D shoot is tricky, get it wrong and you'll have a major headache in post production. With 3D-Depth images saved to a single file as RGB-D (like RGB-Alpha) you only need 1 camera, 1 lens. The image is captured in through a single lens then split using a prism so that one copy goes to a Depth Map censor and the second copy goes to a CMOS or CCD censor. Both images for each censor are then saved in a single file known as RGB-D. Think of editing RGB-D files as editing in 3D with After Effects. We are nolonger editing 2D but true 3D. This type of filming can also be used for future holographic video once mastered.

This may seem a little confusing and I hope I haven't lost you, but I truly believe this is the future for 3D filming that I would like to share. Hopefully the RED team will look in to this for the next generation of RED cameras.

 

[Neil B Smith]

Just to add.

My main line of work is in 3D animation. I use flat 2D images projected on to a 3D object all the time to give the perspective of 3D for that object.

Example: I want to create a 3D model of the Earth. I want to use one of the famously used NASA images of the Earth taken by the Apollo mission. I would create a 3D sphere then planar map the flat 2D image of the Earth on to the 3D object. Voila!! A 3D planet Earth. (After adding atmosphere glow etc..) I can lock together 2 cameras in my 3d software or use an available plugin that will render out 3D image (for use with 3d glasses). I would most likely render out 2 separate camera views then edit in post.

The Depth map will create your 3D modeling, the 2D colour video image will be your paint to the 3D model. What then comes from this is then up to you, either 2D or 3D stereoscopic images. The single perspective is destroyed when outputting to 3D as the software will split itself out from the centerline with 1 camera for left and 1 for right.

I think a Youtube video may be necessary for people to visualise my thinking. You may think I'm way off on another planet! :emote_whack: I'm not, I'm just forward thinking. 2 cameras, or 1 camera with 2 lenses just seem a simple way out to create 3D, its not my thinking. 2 camera filming is not something new. Its like the internal combustion engine, because it works doesn't been we have to stick to it. Lets think out of the box and invent!! :idea:

 

[Neil B Smith]

If you have 3ds max and post DOF then you know the horrendous artifacting that results from trying to use a depth channel to apply depth of field even with a *perfect* depth map on a single layer.

Most 3D depth sensors today are about 320x240 pixels and extremely artifacty. Even if we improve the resolution and reduce the artifacts they won't really be reliable until they can output a perfect 3D scene. If they can output a perfect 3D scene then you're effectively using performance capture anyway.

The future future is performance capture and scene scanning and then actually rendering two eyes. There are just too many artifacts and limitations to using something like even Lidar.

Good points. I actually use 3ds max and know the limitations of Depth maps. 3ds max doesn't do a great job in creating a Depth map but it gives you great control over focus when using either After Effects or Combustion. It saves you from re-rendering the images if the Depth-of-field isn't the way the client wants. I believe it all comes down to how software evaluates the Depth map. After Effects would just blur the image as a 2D image rather than translating in to a 3D image (relief image), then applying depth-of-field blur. Maybe rather than a greyscale image you may need to store per pixel info of Depth. i.e. 800,450,175,50,225..... in a scanline process. You can therefore increase the accuracy beyond the 0-255 limit of a greyscale image, say increasing it to 0-999.

Lidar is an excellent example, thanks for sharing. Its very new tech, just see the possibilities for video and film work.

Getting back to video...
I do believe the technology for Depth map censors are only in their infantcy. Look at TV in the 1920's and look how its today. Now leap a few decades in to the future and see where things will be then. Its just going to be time before it happens.

 

[Jeremy Torrie]

You compare the pictured Sony rig to Bill White's tiny, 67 lb. S1-2K rig (they provided the cams for Saw 3D) it's an easy choice to make. They record uncompressed 2K to a proprietary box, and he's got something like 23 cameras.

I've seen the images first-hand, and they're incredible...and he too believes is true 3D, not this comped 2D stuff.

 

[Dan Hudgins]

Depth Map Sensor

Depth Map Sensor

The image is captured in through a single lens then split using a prism so that one copy goes to a Depth Map censor and the second copy goes to a CMOS or CCD censor. Both images for each censor are then saved in a single file known as RGB-D.

As I understand your quote,

The high resolution image of the "second copy" would not have full detail of the occluded parts of the background.

The "Depth Map Sensor" is an array of low resolution cameras that shoot using various parts of a lens that has a front element about 130mm wide or wider. The low resolution depth images captured off center of the front element will more coma, CA and Asymmetric distortion.

You cannot capture the background parts that are occluded with equal resolution for both eyes and in all respects equal to the other parts of the image with such a system. Nor can the sides of the subject and occluded parts of the background be recorded at high resolution as they would be with two high resolution uncompressed cameras.

The depth accuracy is limited by the interpolation of the so called "depth map sensor" which is a small array of cameras using parts of a common front element.

Fine detail on the surface of the subject would not be recorded in 3D by such a system, as it would be using two high resolution cameras with uncompressed recording, such as two 5K Epic with there "lossless" REDCODE option, perhaps.

What you end up with is something that would record cardboard cut-outs at various depths within the accuracy of the depth recording, but shoot a pole with writing going down both sides pointed at the camera and then try to read the printing on both sides, which you can if you shoot with two cameras.

If you have the single image using a large opening of the single lens, the printing will be out of focus on the sides of the pole pointed at the camera, so you will not be able to map an in-focus version of the writing over the 3D model of the pole in the computer.

In all cases the type of camera you decribe will produce lower resolution on the sides of the subject that using two high resolution cameras where the DOF can be adjusted better.

You can use an array of high-resolution cameras without such a large single lens, and maybe interpolate the 3D depth from those, but you will have high resolution surface depth data, rather than a smoothed and interpolated 3D surface.

The viewers eye-brain can model the 3D depth better than the current type of "depth map sensor" just by looking at two high resolution 3D images and see fine texture on the surface of the subject.

What you are talking about seems to be the Equivalent of the the NTSC system for 3D,

1) You have higher bandwidth and resolution for a "central" single image to map over a lower depth resolution 3D model.

2) The 3D depth data is lower resolution than the image, so motion blur at edges needs to be resolved so that the 3D model does not form odd depth extensions between the background and the foreground.

3) Occluded parts of the background and sides are reporduced with depth that may or may not be right and at lower resolution.

4) The edges of the 3D model may not align with the edges of the high resolution central image that gets mapped over the 3D model.

5) The 3D depth noise needs to be filtered, so the motion and texture of the 3D model could be distorted by low pass filters.

In NTSC the chroma is lower resolution than the Luma, in such a 3D camera, the surface texture and depth are at a lower resolution than the Luma, something that true Stereo 3D using two high resolution cameras would not be afflicted with, and in a two camera setup the DOF of the occluded parts of the sides and bakground that show in the other-eye's-view would look natural, not distorted.

Adjusting a 3D rig is not hard if you take the 3D glasses OFF and look at the stereo pair superimposed. You cannot monitor on a small 3D monitor for projection on a large screen because that tends to have people set the divergence to wide, as divergence on the screen no matter what size cannot be larger than the spacing of the eyes, so its best to have NO divergence so that the images can be viewed on "any" size screen from "any" distance.

Do you have any 4K resolution 3D models captured by such a camera and processed in software that can be rendered without the image mapped over them, of subjects with motion blur? That is like a plaster model on a plaster landscape?

 

[Neil B Smith]

Sony Develops High Frame Rate Single Lens 3D Camera Technology

Sony Develops High Frame Rate Single Lens 3D Camera Technology

http://www.sony.net/SonyInfo/News/Press/200910/09-117E/

Different to what I'm thinking on, but demonstrates that 3D images can be captured using a single lens system.

 

[Dan Hudgins]

Disparity

Disparity

http://www.sony.net/SonyInfo/News/Press/200910/09-117E/

Different to what I'm thinking on, but demonstrates that 3D images can be captured using a single lens system.

I think sony is using that optical way of making sure that there is no disparity in the plane of focus so that the 3D LCD TV sets look sharper, if you have disparity on Lenticular TV sets the parts out of the screen plane do not look as sharp.

Also for the 3D Blu-Ray I think their encoding system works better when there is no disparity in the plane of the focus where the MPEG encoding needs the most bandwidth, having both eyes images the same at those sharp points reduces the difference signal you need to record.

Having that large lens front element makes the camera much larger and heaver than it would be with something like two SI-2K mini.

The same for your single sharp image plus several smaller images over the area of the large camera front element.

With two cameras you can adjust the spacing to be wider for subjects in the distance, but with a single lens camera you are limited to the width of the single heavy large front element and its heavy optics that back it up, so you cannot make a small light weight 3D camera that sees the sides of the subject in the distance with a moderate size single large front element.

You also have the problems of Asymmetric lens aberations from using the parts of the optics that are not paraxal.

With two cameras you can stop the lenses down to get better DOF on the sides of the subject, and both lenses are centered on their sensors so the aberations are symmetric.

==

Your idea of a 3D capture device is useful for some kinds of work, yes, but it is not the best way to capture stereo-3D images to reproduce all the Subtle surface texture that makes viewing 3D images all it can be.

If people only see rounded off and cardboarded 3D rather than lifelike 3D textures, it will not give them the maximum benefit.

When a film producer makes the decision of what way to photograph the subject matter, he should understand the tradeoffs.

 

[Gavin Greenwalt]

. 3ds max doesn't do a great job in creating a Depth map but it gives you great control over focus when using either After Effects or Combustion.

Great control and almost always pretty lousy results from a single layer rendered (which is all you would get from a camera)

Spitting out a single render and applying DOF only works when nothing is occluding anything else to any significant degree.

Similarly if you took a single camera and projected it back onto 3D geometry you then have to *create* everything that the second eye couldn't see that was occluded. The most extreme example would be where one eye sees nothing but fence post and the other eye sees everything else. If your camera only sees fence post your depth map will just see fence post.

You would get horrendous artifacting:

(Super quick and dirty mocked up crossed-eye stereo example rendering.) Aka... cross your eyes to view it.

And that's a "best case scenario" of managing to somehow magically recreate the 3D scene perfectly. With just a depth map it would be 'displaced' and you would have 'smearing' of the 3D geometry.