Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

The best codec for output and archive?

Daniel Browning said:
I recommend Cineform. Visually lossless at moderate filesizes and very well done, IMHO. Costs under a grand.
Click to expand...

Why would anybody even think of choosing a proprietary format for archiving? I can see using proprietary tools for workflow (get the job done right/fast/etc.), but I would expect that when archiving, the most important feature is readability. Will Cineform be prevalent 20 years from now?
 
karapetkov said:
Better than TIFFs? :detective2:

And do you have experience with this 16 bpc codec - Microcosm - by Digital Anarchy [now bought by RedGiant].
Click to expand...

Yes, better than TIFFs. EXR is a float-based format specifically created for high dynamic range (hdr) source material and VFX work by ILM. It is natively supported by NVidia's shader engine for hardware accelerated processing, and as a file format, is much simpler than TIFF. While TIFF is an open standard, in which each vender writes their interpretation of that standard, EXR is open-source (at least in the sense that ILM releases [regularly] their own code to further support it).

I am not familiar with Microcosm.
 
Miraverse said:
Why would anybody even think of choosing a proprietary format for archiving?
Click to expand...

Because some people have different needs.

Some need to archive in a way that will never require further maintenance, like putting it in a time capsule as in the movie "Knowing". For those, it would be vital to use file formats and storage media that are likely to be accessible in the future without any intervention in the intervening 20 years. That might mean petabytes of storage instead of terabytes and making the footage hard to dig up and use.

Other people don't need or want to make it difficult to access, but to have it available at a moment's notice. Cineform files are small and light on CPU utilization, so they can be used with no transcoding or proxies as long as the software is compatible.

Will Cineform be prevalent 20 years from now?
Click to expand...

If any archive format, including Cineform, becomes undesirable in the future, it can be transcoded to whatever new format is desired.

I'd prefer a non-proprietary format, too, but for some needs the compromise is well worth it.
 
Kyle Mallory said:
Yes, better than TIFFs. EXR is a float-based format specifically created for high dynamic range (hdr) source material and VFX work by ILM. It is natively supported by NVidia's shader engine for hardware accelerated processing, and as a file format, is much simpler than TIFF. While TIFF is an open standard, in which each vender writes their interpretation of that standard, EXR is open-source (at least in the sense that ILM releases [regularly] their own code to further support it).

I am not familiar with Microcosm.
Click to expand...


Thanks for the reply.

I am a bit confused whether the 16bit bpc or 32 bpc OpenEXR output is recommended [in Adobe Afx, it says 32 bpc is not recommended].
 
I should take back my previous statement about EXR being 'superior' to 16bpc TIFFs. In all fairness, each has it's selling points... I think I would still use EXR's over 16bpc integer TIFFs for 98% of my work, but there is still that 2%... Anyway, back to your question:

I think 16bpc vs 32bpc depends slightly on the intended outcome/purpose/use of the file. In both cases, EXR's are floating point. Traditionally, 'float'ing point numbers are 32-bit, but EXR utilises a "half" float (16bits). This 16bit-float is the same float format that is compatible with nVidia or another hardware graphics engines. Obviously, saving at 32bpc is more information than 16bpc, but you may lose some performance benefits, with little or no more retained image information. You also effectively double your file size.

The EXR documentation describes 16bpc images as having 30 stops of latitude, and 1024 levels per stop (which is about the same for 16bpc 'signed' integer images). Some precision is lost as a result of the floating point math, in the lower levels of the images, which has the nice side-effect of slightly better results in grain management (since grain is often seen as noise in the lowest levels of the image). This also helps in some of the compression schemes that EXR uses, since grain has a tendency to complicate/reduce compression levels. If you are intent on keeping the integrity of *everything* in your original image, including grain, etc. This might be a reason to keep to TIFFs (that 2% I mentioned), but generally speaking, most people would just as rather lose that noisy garbage anyway.

That said, 32bps is still king for data retention. If you are serious about keeping the highest quality of image, regardless of other factors, I see no reason not to use 32bpc [float] over 16bpc [half] EXRs.

Keep in mind also, that the RED and all other cameras out there, including (by general consensus, film) can't really represent any more than 16bpc of information anyway, sooo.... food for thought.
 
Kyle Mallory said:
I should take back my previous statement about EXR being 'superior' to 16bpc TIFFs. In all fairness, each has it's selling points... I think I would still use EXR's over 16bpc integer TIFFs for 98% of my work, but there is still that 2%... Anyway, back to your question:

I think 16bpc vs 32bpc depends slightly on the intended outcome/purpose/use of the file. In both cases, EXR's are floating point. Traditionally, 'float'ing point numbers are 32-bit, but EXR utilises a "half" float (16bits). This 16bit-float is the same float format that is compatible with nVidia or another hardware graphics engines. Obviously, saving at 32bpc is more information than 16bpc, but you may lose some performance benefits, with little or no more retained image information. You also effectively double your file size.

The EXR documentation describes 16bpc images as having 30 stops of latitude, and 1024 levels per stop (which is about the same for 16bpc 'signed' integer images). Some precision is lost as a result of the floating point math, in the lower levels of the images, which has the nice side-effect of slightly better results in grain management (since grain is often seen as noise in the lowest levels of the image). This also helps in some of the compression schemes that EXR uses, since grain has a tendency to complicate/reduce compression levels. If you are intent on keeping the integrity of *everything* in your original image, including grain, etc. This might be a reason to keep to TIFFs (that 2% I mentioned), but generally speaking, most people would just as rather lose that noisy garbage anyway.

That said, 32bps is still king for data retention. If you are serious about keeping the highest quality of image, regardless of other factors, I see no reason not to use 32bpc [float] over 16bpc [half] EXRs.
Click to expand...


Thanks again for the detailed reply.

I like a subtle amount of grain, but not for all kinds of projects, of course.

So, I guess I'll have to save for more HDDs. :tongue:

Keep in mind also, that the RED and all other cameras out there, including (by general consensus, film) can't really represent any more than 16bpc of information anyway, sooo.... food for thought.
Click to expand...

That is good to know. I also read somewhere that the nuances found in 16bpc are ~ like the ones the human eye can see.

So, that's the limit alright.

Merry Christmas! :sorcerer:
 
Macgregor,

I was trying to find a good way of explaining the benefits, but ultimately decided to let nVidia & GPU Gems do it instead. From Chapter 26 of GPU Gems:

26.1.3 Range of Representable Values

The most obvious benefit of half is the range of values that can be represented with only 16 bits. We can store a maximum image value of 65504.0 and a minimum value of 5.96–8. This is a dynamic range of a trillion to one. Any image requiring this range would be extremely rare, but images with a million-to-one range do occur, and thus they are comfortably represented with a half.

Photographers measure dynamic range in stops, where a single stop is a factor of 2. In an image with five stops of range, the brightest region is 32 times brighter than the darkest region. A range of one-million to one is 20 stops in photographic terms.
26.1.4 Color Resolution

The second, less obvious benefit of half is the color resolution. Each stop contains 1024 distinct values. This makes OpenEXR excellent for normal-dynamic-range images as well. In an eight-stop image that ranges from 1.0 down to 0.0039 (or 2–8), the half format will provide 8192 values per channel, whereas an 8-bit, gamma 2.2 image will have only 235 values (21 through 255).
Click to expand...

And to answer the question of compression formats:

26.3 OpenEXR Data Compression

OpenEXR offers three different data compression methods, each of which has differing trade-offs in speed versus compression ratio. All three compression schemes, as listed in Table 26-2, are lossless; compressing and uncompressing does not alter the pixel data.
Table 26-2. OpenEXR Supported Compression Options

Name


Description

PIZ


A wavelet transform is applied to the pixel data, and the result is Huffman-encoded. This scheme tends to provide the best compression ratio for photographic images. Files are compressed and decompressed at about the same speed. For photographic images with film grain, the files are reduced to between 35 and 55 percent of their uncompressed size.

ZIP


Differences between horizontally adjacent pixels are compressed using the open-source zlib library. ZIP decompression is faster than PIZ decompression, but ZIP compression is significantly slower. Compressed photographic images are often 45 to 55 percent of their uncompressed size.

RLE


Run-length encoding of differences between horizontally adjacent pixels. This method is fast, and it works well for images with large flat areas. However, for photographic images, the compressed file size is usually 60 to 75 percent of the uncompressed size.

Optionally, the pixels can be stored in uncompressed form. If stored on a fast file system, uncompressed files can be written and read significantly faster than compressed files.
Click to expand...
 
Radoslav Karapetkov said:
I am a bit confused whether the 16bit bpc or 32 bpc OpenEXR output is recommended [in Adobe Afx, it says 32 bpc is not recommended].
Click to expand...

OpenEXR supports both 16-bit float and 32-bit float. The reason 32-bit is "not recommended" in After Effects is that it's almost never necessary. Many people were checking the 32-bit option, thinking they were getting some better image quality. Or maybe they thought they had to check it to get float, because standard IEEE float is 32-bits. What they probably got instead was a 2x bigger file with no improvement in image quality. Even worse, some programs using older versions of the OpenEXR library might not be able to read those files.

The maximum value for a 16-bit float is 65504.0 (where white is 1.0). That's 16 stops brighter than normal white! If you actually have values higher than that, then you have a reason to use 32-bit float. But even HDRs with the sun in them don't usually get close.


Brendan
 
Last edited:
You must log in or register to reply here.
Back
Top