The next 4K digital mag for Aäton Penelope described by JP Beauviala! [english subt.]

[Deanan]

Yes, they do. But that doesn't necessarily mean you can't use it in a different way, especially if there is no other file format that is particularly suited for the particular need. DPX as a format has a lot of room for both essence and metadata, and it is a file that by definition defines one frame, as opposed to a wrapped movie file. It is also recognized by a lot of copy, backup, and SAN software on various platforms as a format that implies an image sequence, allowing sequence aware utilities to do their work efficiently. So rather than invent a completely new file format, using an existing one as a container has some advantages.

This was an issue at dalsa also where the raw was saved in a dpx file. It's basically no different that putting a dpx header on any random data. You can do it, but everyone else needs to support your format specially and most don't even support the dpx spec properly. You run into issues like bayer pattern order, etc.

 

[M Most]

...everyone else needs to support your format specially and most don't even support the dpx spec properly. You run into issues like bayer pattern order, etc.

I don't see how that's solved by going to an entirely new, proprietary format. You still need to have everyone support a format that nobody else is using, and if you're using a unique extension, existing software is not necessarily aware of the sequential nature of the files. I suppose DNG is one possibility, but the industry doesn't seem to be rushing to support that, either. At least with DPX there's a general understanding of the placement of metadata and file essence, even if it still has to be interpreted by a demosaic routine.

 

[Mitch Gross]

I'm with Mike Most on this one. The metadata header is enough to tag what the file is and then there are plenty of deBayer systems that will be able to get right to work on these files. I'm sure that GlueTools, Iridas and a host of oters could work such files into their systems with little headache. Yet by remaining in the DPX file structre it makes transport of the stream far easier, and I'll vote for storing RAW over the 3X-sized RGB any day. So much more efficient.

 

[Deanan]

I don't see how that's solved by going to an entirely new, proprietary format. You still need to have everyone support a format that nobody else is using, and if you're using a unique extension, existing software is not necessarily aware of the sequential nature of the files. I suppose DNG is one possibility, but the industry doesn't seem to be rushing to support that, either. At least with DPX there's a general understanding of the placement of metadata and file essence, even if it still has to be interpreted by a demosaic routine.

By making a non-standard dpx, you're essentially creating a proprietary format with the illusion of being standard. DPX has enough confusion and messed up implementations around the standardized part of the spec as it is.
When you start adding up things like non-standard audio in user data, misuse or unclear usage of many of the header fields fields, odd packing formats not in the standard (and the odd ones in the standard , etc. it just makes it more of a headache for developers to support dpx properly. Especially since there is no reference implementation.

 

[Pawel Achtel]

JP told a lot more publicly than I thought he would ...

You guys are gonna love this. By the time this rolls around, Uncompressed RAW will not look so scary...

Let me guess, 2020? :rofl:

We discussed this many times over: the very success of RED One is because of good compression and very high resolution. Phantom is not really the best camera for shooting and storing tens/hundreds of hours of footage. The image quality coming from MX is way higher than that from the Phantom and at a fraction of the bandwidth/storage requirements. Phantom is good for its primary purpose: high speed. I wouldn't shoot a feature or a documentary on it.

 

[Graeme Nattress]

I don't see how that's solved by going to an entirely new, proprietary format. You still need to have everyone support a format that nobody else is using, and if you're using a unique extension, existing software is not necessarily aware of the sequential nature of the files. I suppose DNG is one possibility, but the industry doesn't seem to be rushing to support that, either. At least with DPX there's a general understanding of the placement of metadata and file essence, even if it still has to be interpreted by a demosaic routine.

Well, if they did do DNG they'd get a two-for-one in being able to handle the files from the Ikonoscop also. And they'd be able to run the images from the camera through After Effects.

With every decision there are pros and cons... And no word on the bit depth of the DPX? Many apps don't support anything other than the 10bit DPX.

Graeme

 

[Tom.Wong]

Hi,

I agree with Mitch, some numbers here are simply blown out of proportion!

On the video, when I titled that uncompressed 4K 10bit 444 @ 24im/sec represents a 750MB/sec bandwidth (actually, I should have been more accurate, the precise number is 759.375MB/sec), here is how I calculated it: 3*4096*2160*10*24/8/1024/1024 = 759.375MB/sec
3 for 4:4:4
4096 and 2160 for the official 4K format
10 for 10bit
24 for 24 im/sec
/8 to get Bytes
/1024 to get kBytes
/1024 to get MBytes
So, I may be wrong in my calculation way but to me,
uncompressed 8bit 4K 444 @ 24im/sec is 607.5MB/sec not 875.25MB/sec
and uncompressed 12bit 4K 444 @ 24im/sec is 911.25MB/sec not 1.3GB/sec
...I agree: that's still gigantic! Even with a few technical 'tricks' to reduce the flow. And knowing that JP Beauviala wants to propose a 50im/sec maximum frame rate for his digimag, meaning 1.545GB/sec of 10bit bandwidth, I understand why the SSDs will be 0 raided.
Gigantic, indeed, but not unreachable anymore.

SSDs have been making huge progresses last past few years and now make it possible to build a handheld and ergonomic recording solution.
In my view, the maine issue of 4K workflow has always been 'how to record such a flow in an ergonomic way on the set?' much more than 'how to manipulate it in post?'.
And who talks about editing uncompressed 4K on-line?
We definitely learned something about the way we've been editing 16 and 35mm film with computers for more than 25 years, right? So, do we need 4K dailies in the editing-room? I guess not. 1K proxy dailies are good enough for editing.
And, even if you want to edit uncompressed 4K, solutions are on their way.

http://www.tomshardware.com/news/ocz-ssd-soild-state-terabyte-madshrimps,7200.html

case and point, if this is already at the "consumer" level, a year from now, where will it be at in the professional world?

 

[Tim Morten]

http://www.tomshardware.com/news/ocz-ssd-soild-state-terabyte-madshrimps,7200.html

case and point, if this is already at the "consumer" level, a year from now, where will it be at in the professional world?

Let's look at some quotes from the article you linked to:

1. "Madshrimps fired up SiSoft Sandra File system and found that the Z Drive outputted a drive index of around 550 MB/sec."

This is not fast enough for even a single 4K stream.

2. "There's no word on availability or final pricing for the drive/device/array/beast. "

So it's a tech demo of a non-shipping un-priced device.

Not consumer level quite yet...

"Two years" is a popular answer when people don't really know how long it will take for technology to get there .

http://www.telegraph.co.uk/news/new...errari-could-be-reality-within-two-years.html

Cheers,
Tim

 

[Tom.Wong]

the point is that it's just an example of how it's getting there, professional industry tech is custom made and blows a lot of consumer level stuff out of the water. if something this fast is at the consumer level already, how much faster is it gonna get in a year from now? 6 moths ago your normal SSD could give you about 220/mb/sec, this is about double, another 6 months, 1.1 gb/sec? 6 months after that, 2.2? 3? it's going to get there, faster than anybody will anticipate.

 

[Tim Morten]

the point is that it's just an example of how it's getting there, professional industry tech is custom made and blows a lot of consumer level stuff out of the water. if something this fast is at the consumer level already, how much faster is it gonna get in a year from now? 6 moths ago your normal SSD could give you about 220/mb/sec, this is about double, another 6 months, 1.1 gb/sec? 6 months after that, 2.2? 3? it's going to get there, faster than anybody will anticipate.

Again, your starting point is an unreleased product that still won't be fast enough to handle a single stream of uncompressed 4K, and whose price we do not know. So I maintain that it is NOT at a consumer level yet, and it's unclear that even this first product will be available in 6 months. How long is it before the product you linked to is even released? How long before it's actually fast enough to handle 4K? How long before it's at a consumer price-point?

I'm not disputing whether the tech will get there. I am disputing whether it will get there with a cost and timeframe that will make it accessible to me. If you think we're six months away from 1.1gb/s SSD's that are at a consumer price point, I've got a bridge to sell you... or a Carrion monitor with a Viewfactor wireless follow focus .

I absolutely believe that for an exhorbitant price, you can create an SSD RAID with sufficient bandwidth and capacity right now. I also believe that it will take at least half a decade before that price comes down to something that makes it accessible to me.

The high cost/long time-frame/lack of established visual improvement over what we already have combine to make me skeptical of the merits of an uncompressed 4K approach. I think it will be 5 years or more before it is practical (and I can't WAIT to see where Red has gotten to in 5 years).

I'm sure there will be folks that drop $80k on a Penelope body/digital mag when it releases... as Jim said, more choices for the customer. In my case, I plan to continue choosing tech that is practical, and that I can afford.

Cheers,
Tim

 

[Thomas Chatelet]

Well, in my view, 'more choices for rental houses and some healthy production companies' would be a more realistic statement if you're aware of what really matter with Aäton products.

Penelope is beginning to make a name of herself: in one year, many shootings around the world (mainly in 2perfs 35mm), lots of enthusiasm and positive feedbacks from operators and cinematographers and rental houses are buying her to be able to propose her to more costumers at the same time.
Does she compete with RED One? Well, in rental houses catalogues only. As with ARRI products, consumers aren't the target of Aäton products (except for Cantar).

So, what is really at stake with Penelope and her uncompressed 4K digital mag?

- For the set, being an ergonomic, hybrid and flexible camera.
So, if you want to mainly shoot your film in 4K for economical, pragmatical, enthusiastic, whatever reasons, you know that you'll also be able to efficiently respond, on the set, to extreme situations of shooting (big heat, great cold, high contrast of lights, etc.) where 35mm has been proven itself for more than a century.
Penelope will be able to switch on the set a 4K mag with a 35mm mag, and vice versa, in less than 10 minutes!

- For rental houses, well, they just have to buy a "plug and play" 4K digital mag for the Penelope bodies they already have. A pretty good investment, I guess.

- For editing rooms and postproduction providers, like I said, there is absolutely no need to edit uncompressed 4K, 1K proxies are enough. Backups of dailies et downconversions in 1K, made directly on the set, are not a hard thing to ergonomically create and set up.
Yes, a DPX format with relevant chunks remains to be set up in order to work in a efficient workflow but, like JP said, 'nothing unreachable'.
And after that, what else?
Can I edit 1K proxy dailies on my favorite high-end MacPro workstation? Yes.
And do we have the technology to store, conform and grade uncompressed 4K in professional postproduction facilities? Yes, mostly, and 2 years from now, I bet all of them will be able to provide it.

So, to me, the question is less 'when workstations' components handling uncompressed 4K will be at a consumer price level?' than 'aren't Penelope's ergonomics and a good DPX workflow just what the industry needs for uncompressed 4K feature productions?!'

 

[Graeme Nattress]

But it's not a "DPX workflow" - it's just using DPX as a wrapper for raw bayer pattern data at to be determined bit depth. It's not providing you with the equivalent of a 10bit log film scan, which is what, for many, DPX workflow means. It could be shooting a bunch of .P4K files for all it's relevance to a traditional DPX workflow.

Graeme

 

[Cüneyt Kaya]

in times with wavelet compressions i dont quite the see advantage of UC 4k, except VFX shots maybe.

but i am ready to learn

 

[Mitch Gross]

Let me guess, 2020? :rofl:

We discussed this many times over: the very success of RED One is because of good compression and very high resolution. Phantom is not really the best camera for shooting and storing tens/hundreds of hours of footage. The image quality coming from MX is way higher than that from the Phantom and at a fraction of the bandwidth/storage requirements. Phantom is good for its primary purpose: high speed. I wouldn't shoot a feature or a documentary on it.

We currently have a major production shooting on a pair of Phantom 65 cameras. You cannot imagine how beautiful such an image is until you see it. You're comparing to Phantom HD, and I have no idea if you have seen the best implementations of that camera or not. I will hold Phantom performance out there with the best of 'em.

 

[Tim Morten]

More choices is always a good thing.

It's reasonable for us to expect that Penelope is going to require a substantial price premium over Red, due to the requirements of its uncompressed acquisition format (and Aaton's price history in general).

It's also reasonable for us to expect that the only practical way to work with Penelope footage will be a return to transcoding (even if the DPX generated were debayered, no mainstream users will be able to realistically work with uncompressed footage for years to come). Some are still using a transcoded workflow now, so no loss to them -- those of us who are using Red's "plug-and-edit" workflow will be less enthusiastic to lose that convenience.

It's possible that Penelope will offer some measurable improvement in image quality to justify the higher cost and inconvenience... for Aaton's sake, I hope so. We'll have to wait and see on that one.

The other benefits noted (switch between 35 and digital, allegedly greater physical robustness than Red) seem of questionable value relative to what will surely be the cost differential. Maybe there's a niche market there for directors who don't want to give up film, but need digital for certain VFX shots. I'm not in that category... I have no qualms shooting digital all the way.

As I said earlier, I admire Aaton as a company (and I obviously understand Mitch's enthusiasm as a dealer), but I'm very skeptical about the uncompressed approach. Best-case, it becomes an expensive option to gain some undetermined visual improvement. Worst-case, it's an unnecessary price premium that doesn't generate meaningful visual improvement. Time will tell...

- Tim

 

[Thomas Chatelet]

It's not providing you with the equivalent of a 10bit log film scan (...)

Why not? We'll see at NAB (if somebody else ask JP Beauviala 'cause I won't be there) but I wouldn't bet my house on that one!
Let's be clear, this is just pure speculation from my part but I wouldn't be surprised at all if it reveals to be the chosen option.
It's just the expression you used. A 10bit log film scan. I think it's just where JP Beauviala is planning to get at: to capture digital informations as close as a 10bit log film scan. To me, and knowing a bit Beauviala's philosophy, it just makes sense.
But, like Tim said, time will tell.

 

[Graeme Nattress]

Thomas, JP surely could do that if he wants, and it also ups the data storage requirements 3x too

Graeme

 

[Dan Hudgins]

delta-encoding and bandwidth

delta-encoding and bandwidth

4k bandwidth for 100% true RAW,

for 16:9 12bit (14bit can be supported by the same delta-encoding)

4096x2304x1.5x24=340MB/s *0.7 = 238MB/s average 340MB/s peak, 170MB/s low.

for 1.85:1 12bit (14bit can be supported by the same delta-encoding)

4096x2214x1.5x24=327MB/s *0.7 = 229MB/s average 327MB/s peak, 163MB/s low.

for 2.35:1 12bit (14bit can be supported by the same delta-encoding)

4096x1742x1.5x24=257MB/s *0.7 = 179MB/s average 257MB/s peak, 128MB/s low.

The values for 2.35:1 can be done with one memory module (maybe, peak bandwidth is not something that you get because if your over or under expose the bandwidth goes down most of the time), and the other rates with two as 440MB/s with two is going to be here soon, probably.

These values are not "pie in the sky" but are close to what as been done already.

As for the file format, they should use DNG like Acam dII (packed to 12 bits, the SI-2K and Kinor-2K sample DNG I have did not pack the bits for some unknown reason?), it will become part of the CinemaDNG standard and supported by many programs. I have my DNG working and delta-encoding for smaller file sizes to archive. So called "Linear DNG" (its a TIF file RGB not white balanced, not a Bayer pattern DNG that is not interpolated) is not a good idea for Bayer data since it locks in the defects of the interpolation used, sensor data should be archived UN-interpolated since the way it is interpolated changes the outcome in many ways.

I've been working on my workprint conversion from true RAW sensor data to my PIX playback frames that are used for editing, and have been able to get conversion rates of better than 1fps on my slow computer, on a faster computer conversions of 5fps or more should be possable, so converting the camera files to playback frames with color correction for editing should not be a major issue for filmmaking. You can then do the full de-Bayer and CC overnight+ to produce the 4K TIF for the film recorder or DCI etc.

For people who want just 24fps for filmmaking a simple module for Scarlet with a true RAW output connector does not seem like such a complex thing to make, and seems like it would be easer than the other modules you have made already...

You don't want sensor data Linear in less than 12bits because after the shadows are pulled up you NEED the lower bits to average the so called noise into a signal. 10bits gamma or LOG is not enough to give wide grading range and end up with the needed 9bits of good data. 14bit delta-encoding (with 14bit packing for over-flow frames, although you could drop the bottom two bits for that case since the frames would have to have most of the data in the highlight end of the scale in order to overflow the bandwidth cap for delta-encoding) is the best what to go for linear sensor data and have enough "balance" to the noise after grading to average down the right tone values. Delta-encoding is 100% lossless, the bandwidth cap is just to avoid frames using more bandwidth than bit-packing would, bit-packing is also 100% lossless, there is no need to put sensor data into 16bit per pixel now since there are no sensors that have enough signal at the bottom end to get much out of that, maybe in a few years but not anything I have seen so far needs 16bits per pixel... after de-Bayer 16bits in a TIF is useful to reduce scaling errors with grading and filters that work on the same image over and over...

 

[Dan Hudgins]

store, conform and grade uncompressed 4K

store, conform and grade uncompressed 4K

And do we have the technology to store, conform and grade uncompressed 4K in professional postproduction facilities?

Knock, Knock... you can do it at home now. (look down)

 

[Dan Hudgins]

Price

Price

I absolutely believe that for an exhorbitant price, you can create an SSD RAID with sufficient bandwidth and capacity right now. I also believe that it will take at least half a decade before that price comes down to something that makes it accessible to me.
Tim

What would be a "fair price" to record uncompressed before the end of 2010?