SSD: more $$$ Confusion ????

[James Brundige]

It is the nature of the beast, the more we get the more we want, just to see if we can get it, and because we are Gluttons.
I know it sounds Ridiculous but would not it be Cool to record 1:1 from epic?

What's cool about it? Sounds like more storage, more bandwidth, more headaches, without a visible difference. Redcode is the software foundation that made the Redvolution possible.

 

[Albert Astorga Zepeda]

you are right

you are right

What's cool about it? Sounds like more storage, more bandwidth, more headaches, without a visible difference. Redcode is the software foundation that made the Redvolution possible.

you might be right, but I know if enabled someone will do it. and maybe for a real project. I would do it just for fun.

 

[Detlev Eller]

VFX people have said that they have never had better footage to work with than the RED ONE M-X (at 7.5:1). Why would they need 3:1? Just curious...

Jim

I have no hands on experience with the footage,
but theoretically speaking ....

Evolution.
Error tolerance in anything procedural.
Once we have it (1:1), we will build on it.
I like the idea of native, it eliminates one variable.

In very complex image layering and manipulation and nested compositions (as in Photoshop/After Effects),
compression artifacts may build up, theoretically speaking.

 

[Deanan]

It is the nature of the beast, the more we get the more we want, just to see if we can get it, and because we are Gluttons.
I know it sounds Ridiculous but would not it be Cool to record 1:1 from epic?

Not cool when you consider the data rates go up to multiple GB/s and especially when 2.5:1 mathematically lossless is exactly the same image.

 

[Detlev Eller]

Not cool when you consider the data rates go up to multiple GB/s and especially when 2.5:1 mathematically lossless is exactly the same image.

... 100% the same image, when looking at the image as a whole,
but what if looking at it on the pixel level for compositing math or some clever upcoming image analysis?

theoretically speaking,
Just exploring here ...

 

[IAN SUN]

As far as my storage system is concerned, RedCode is cool.
Uncompressed, not so much.

 

[IAN SUN]

... 100% the same image, when looking at the image as a whole,
but what if looking at it on the pixel level for compositing math or some clever upcoming image analysis?

theoretically speaking,
Just exploring here ...

Mathematically Lossless, so yes all data and therefore every pixel should be identical to the uncompressed.

 

[Detlev Eller]

Mathematically Lossless, so yes all data and therefore every pixel should be identical to the uncompressed.

Ian,
Mathematically lossless is not the same as lossless.

In a lossless situation, the data lies on a kind of grid in the file,
whereas in a mathematically lossless scenario, the information is described in an algorithm ....


example:
Algo: 5 x [black pixel]
lossless raw Image info: [black pixel] [black pixel] [black pixel] [black pixel] [black pixel]


The code to manipulate this data is different.

I can only repeat,
this is a theoretical discussion, since Jim raised the question of "why would VFX guys" want it.

VFX people have said that they have never had better footage to work with than the RED ONE M-X (at 7.5:1). Why would they need 3:1? Just curious...

Jim

The difference between a 7.5:1 compression and 3:1 may be, theoretically speaking, a speed benefit in decompression. The implementation of 1:1 RAW may open up new possibilities.

In a world where "Storage" space is not the constraint,
this is the answer. It would save time or enable stuff, it may speed up workflow ... or other stuff, that will come when we´ll have it.

Of course, we can always transcode from a "mathematically lossless" aquisition format to a "lossless format", which may speed up the rendering of complex compositings .... for example.

The extra price is transcoding time and fast storage.

I plead, everything is subject to test and measure, there is no general rule for all and any circumstance.

 

[Dominic Jones]

Ian,
Mathematically lossless is not the same as lossless.

That is incorrect. The very definition of "mathematically lossless" is that after decompression the exact same data as went into compression is retrieved.

Examples of mathematically lossless compression are PKZip (.zip) RAR (.rar) and StuffIt (.sit) files - these are all used to compress (among other things) executable files, which believe you me, definitely need to be bit-for-bit replicas of the original!

Now, "visually lossless", that's a different story. But with mathematically lossless you do not lose any data whatsoever, by definition.

Cheers,
Dom.

EDIT: After re-reading your post, I see that you're talking really about the method of storage. Bear in mind that the data is not used in it's compressed state - it is decompressed to exactly the same state as you describe (i.e. the original data state) before use. Often this might be internally in some program or another, but it will always be the case. There is no advantage (other than perhaps processing speed) to uncompressed (which I believe you are referring to as lossless) vs mathematically lossless compression...

Since you mention speeding up VFX processes as perhaps being a benefit of uncompressed vs ML codecs, bear in mind that it's not just size of storage that counts in this window of reference, but speed of moving data. Your transfer speeds would need to be high enough (from the disk array, through the FSB of your machine(s), and into and out of page-files, as you're not going to have enough RAM to work with paging) to offset the decompression time per frame. At present, that would take one hell of a system, and I would posit that for the immediate future that will remain the case. Of course, I could be wrong!!

 

[Dwain Barrick]

Jim,

How will digital stills files be saved. Traditionally they are saved to CF and or tethered to a computer. Will CF still come into play for this mode or will the SSD be the new capture standard for stills on the Epic?

Asking this question, makes me think of another. Will the Epic integrate seamlessly with current Stills capture software ie. Capture One or Aperture?

 

[Steven Caesare]

Ian,
Mathematically lossless is not the same as lossless.

In a lossless situation, the data lies on a kind of grid in the file,
whereas in a mathematically lossless scenario, the information is described in an algorithm ....

...SNIP...

You are confusing "lossless" and "compressed".

Mathematcially lossless IS lossless. You have not lost a single bit of data.

Now compressed can be lossy, or lossless. Redcode at 8:1 is lossy compression. Redcode at 2.5:1 is lossless.

As for how the data is stored... that's completely irrelavant. You do recognize tha the physical layout of bits on the SSD will be different than the layout on your editing workstation, which will in turn be different than the layout inside your archival SAN, etc...

It's making sure the same stream of bits that goes in to the file come backs out is what matters.

--sc

 

[Detlev Eller]

... There is no advantage (other than perhaps processing speed) to uncompressed

... processing speed is exactly the point.

Reducing de-compression (aka translation) computing cycles will speed up situations with complex VFX workflows.

In VFX, speed is everything.
Waiting time is counter productive and expensive.


Sorry guys, regarding the lossless term confusion,
i should have said "Mathematical lossless is not Native"

How data is stored, is fundamental for speed.
As Red proves this point.

However, i am thinking in code here, which works with files ... not images we see in the application GUI ... ;-)

Less complexity => Faster render => faster results.

 

[Ivan G]

Call me crazy but I want to capture in the least compression possible. VFX or not I want uncompressed! I guess as years went by and watching previous projects shot in the 80's and 90's on different formats would of been amazing to see what it really looked like in 5K. I know in 20 years from now it will be nothing compared to 30K but still some serious quality. I'm a quality whore, sorry.

 

[David Rasberry]

... processing speed is exactly the point.

Reducing de-compression (aka translation) computing cycles will speed up situations with complex VFX workflows.

In VFX, speed is everything.
Waiting time is counter productive and expensive.


Sorry guys, regarding the lossless term confusion,
i should have said "Mathematical lossless is not Native"

How data is stored, is fundamental for speed.
As Red proves this point.

However, i am thinking in code here, which works with files ... not images we see in the application GUI ... ;-)

Less complexity => Faster render => faster results.

Regardless of compression or lack of it in the raw acquisition file, for most VFX work it will have to be deBayered to some form of uncompressed RGB file first anyway won't it?

 

[Steven Caesare]

... processing speed is exactly the point.

Reducing de-compression (aka translation) computing cycles will speed up situations with complex VFX workflows.

In VFX, speed is everything.
Waiting time is counter productive and expensive.


Sorry guys, regarding the lossless term confusion,
i should have said "Mathematical lossless is not Native"

How data is stored, is fundamental for speed.
As Red proves this point.

However, i am thinking in code here, which works with files ... not images we see in the application GUI ... ;-)

Less complexity => Faster render => faster results.

I think you are trying to make the point: By not having to spend CPU cycles in ordter to de-compress files (even if mathematically lossless), then you won't wast time doing VFX renders

If that is your point, then that's not necessarily true.

The fact is that many operations (especially video work), can be I/O bound. Moore's law has kept CPU power pushing ever forward, and while disk storage SIZES have gotten progressivley larger, disk I/O speed has not nearly kept pace.

Hence in many cases, you can actually GAIN performance with compressed data. It takes less I/O time to transfer the smaller data set and then decompress it with your CPU, then it would take to transfer 3X the data from the disk.

We see this with our servers: compressing data actually speeds up perfomance, provided we have the CPU power to do it.

-sc

 

[D Fuller]

... processing speed is exactly the point.

Reducing de-compression (aka translation) computing cycles will speed up situations with complex VFX workflows.

In VFX, speed is everything.
Waiting time is counter productive and expensive.

...

Less complexity => Faster render => faster results.

That would be true if the processor were the only component of the system, but at the data rates you would be working with, the time required to move that much data in and out of memory would likely be be more significant than the time savings from dealing with a lossless compression algorithm.

 

[Jeff Kilgroe]

That is incorrect. The very definition of "mathematically lossless" is that after decompression the exact same data as went into compression is retrieved.

Precisely. I think he's confusing "mathematically lossless" with "visually lossless".

I think RED is driving home the point of "mathematically lossless" because they have not truly delivered that yet with current REDCODE formats, so this is NEW!!! And also because many of the so-called lossless image formats and video codecs out there are truly only lossless as it applies to general visual inspection, but would fail a mathematical comparison.

 

[Curran Giddens]

... processing speed is exactly the point.

If debayering .R3D is taking up too much of your CPU cycles I suggest buying a Red Rocket card. There are many software programs that now take advantage of it. Which programs do you use?

 

[Ryan E. Walters]

So, if the CF module was originally part of the Stage 2 program and the SSD module is going to be the same price as the CF module, then does that mean that the SSD module will now be included without a price increase for stage 2? So all we would have to buy is whatever media we need?

Any chance on getting an answer to this?

 

[Curran Giddens]

Any chance on getting an answer to this?

Jim answered in another thread. It is "too early" to say for sure. But that it will "likely" be the case.

You guys are ignoring the "too early" and "likely"...

Jim