Digital Cinema Package

[fabrizio carraro]

2Wolfgang Woehl, Danke schön!
It works, and it works perfect. I'd also want to add that if anyone would use these tools. Never use adobe products for generating or converting tiff's. It adds specifics tags in tiff, that couldn't been recognized by other programs (e.g. image_to_j2k). Also this file image_to_j2k you should take from the original OpenJPEG200 package, the opencinematools' file contains a tiff related error (at least now 2009-12-19 version 1.1.2). Looks like we've mentioned all the rocks on this "bumpy road". Questions are welcomed. This information is based on experiments with Dolby DSS-200 server. I hope it's also right for other servers, but i don't realy know
And the first question is mine. What about generating content. We used 3D max's tiff sequence, generated by 2 cameras. But we couldn't manage flying outside effect. I mean an effect when some object is hanging in the middle of the cinema hall. Can anyone give me advice, information or, i, don't know, book, tips and tricks of stereoscopic content ? Or if you generate live content with 2-cameras equipment (that's more probable, see forum name ). What equipment can i use for stereoscopic shooting on semi-pro level?

I compared opencinematools with easyDcp. Using rgb to xyz conversion in "convert -recolor" with the same matrix mentioned above (0.4124564 0.3575761 0.1804375, etc) I got a bit less warm and less saturated colors. Then, checking on Bruce Lindbloom's matrices page I found out that that matrix is a sRGB to XYZ matrix, so no wonder why the colors were a bit different. I don't thing that as a DCP mastering tool easyDcp uses sRGB, but I have to admit that I don't know which one is the right one!
Apart from that the combination imagemagick, openjpeg and opencinematools works perfectly (even if the tiff to jpeg conversion takes ages. How did Fraunhofer managed to make it so fast?)
Furthermore: anybody managed to generate an encrypted DCP and its KDMs with opencinematools?

 

[Mike Mazzotta]

I'm surprised nobody has brought up the REDRay yet. . .


I heard that Redcode & DCP are VERY similar. . . and that encoding DCP from R3D files will become easier and easier. . .

I ALSO heard that the Pro version of the REDRAY will replace MANY of the DCP servers in movie theaters, because of cost. . . and with all those new Sony Projectors going up on Regal Screens, I think that RED will completely dominate that market. ..


Of course. . . that's just ALL rumor and conjecture.

 

[Jean-Charles Wolfarth]

Pavel & Wolfgang,

It would be great if you could release a small tutorial which describes all the steps to get a DCP package with the Opencinematools... :smile5:

I know, I know, it's a little bit of work, but I'm sure there would be plenty of people out there who would be grateful... well, at least me, I would owe you a glass of wine... :smile5:

 

[Wolfgang Woehl]

Jean-Charles,
like I said: opencinematools is merely a frontend to a number of tools plus 3 tiny apps to create CPL, PKL, ASSETMAP and VOLINDEX. The windows version of opencinematools carries all the necessary tools along and offers a rudimentary gui. So for the sake of transparency here are all the required steps that you would take on a linux box in order to create a DCP.

1. Think about what kind of package you want.
A "cinema" package (24/48fps/JPEG2000-encoded picture essence 4:4:4/12bit per channel/X'Y'Z'/Sound 48khz/24bit)?
Or an MXF Interop (25fps/50i/60i/MPEG2-encoded picture essence 4:2:0/8bit/YCbCr-encoded/Rec.709 or whatever/Sound 48khz/24bit)?
The main differences being picture codec, colorspace and framerates.

2. Prepare the source.
"Cinema" package: At the end of this step you want to have a directory of whitepoint-adjusted, gamma-corrected, colorspace-converted, scaled and bit-padded tiffs that you will throw at the JPEG2000 encoder.
There's a gazillion ways to create an image sequence from your source material. Like "mplayer source -vo pnm" or "ffmpeg -i source -f image2 -vcodec tiff %06d.tif" or export from your editor. Depends on your source and to what extent the used tool lets you control its output.
On to scaling, bit-padding (12 bits per color channel required), gamma and whitepoint adjustment, colorspace conversion. Again, there's a lot of tools which can do this. An ImageMagick example for the sake of brevity:

cd tiffs;
for f in *tif; do
convert $f
-alpha Off -resize 2048x1080 -depth 12 -gamma 1.181818
-recolor
"0.4124564 0.3575761 0.1804375
0.2126729 0.7151522 0.0721750
0.0193339 0.1191920 0.9503041"
../tiffs-final/$f;
done

gets rid of a possible alpha channel, scales, pads bit-depth to 12, adjusts gamma by 1.181818 (not "sets to") and converts from sRGB to XYZ (look up http://www.brucelindbloom.com/Eqn_RGB_XYZ_Matrix.html for colorspace conversion matrices). You would use another recolor (after colorspace) for whitepoint-adjustment (look up http://www.brucelindbloom.com/index.html?Eqn_ChromAdapt.html for matrices) if needed.

MXF Interop: You want an MPEG2 elementary stream to stuff into MXF, no need for an image sequence in this case. If, for example, you have an interlaced transport stream with pixel aspect ratio 1.333 you can use ffmpeg to say
ffmpeg -i source.mts -flags +ildct+ilme -s 1920x1080 -an -sameq final.m2v
This is actually transcoding the source so you want to make sure no funky colorstuff is happening along the way. I think this is missing a pixel aspect ratio flag, too. We'll find out in due time (Had nothing but trouble with 1.333 pas on servers/projectors so that's the reason for the rescale). -an ignores audio (You want to bit-pad and resample audio from sox or any DAW. Basically you need discrete mono/48khz/24bit files. Call them something like 00_left.wav, 01_right.wav etc.)

Oh, my daughter is waking up, need to go, the other steps this evening.
Wolfgang Woehl

 

[fabrizio carraro]

Tina,

I have some doubt about the gamma conversion. The way other tools (for example easydcp) make the xyz conversion is: degamma 2.4, rgb->xyz, re-gamma 2.6
Even if at the end of the chain the gamma is still changed by 0.2 (very close to 0.1818) the conversion to xyz is done on different values. Does this change the final result? Furthermore, the matrix is a sRGB to xyz matrix, but which is the official DCI matrix? I was not able to find it out.

> convert $f
> -alpha Off -resize 2048x1080 -depth 12 -gamma 1.181818
> -recolor
> "0.4124564 0.3575761 0.1804375
> 0.2126729 0.7151522 0.0721750
> 0.0193339 0.1191920 0.9503041"

Fabrizio

 

[Wolfgang Woehl]

Fabrizio, wrt gamma correction: there is a difference (and it's not marginal) between degamma -> colorspace conversion -> gamma and colorspace conversion -> gamma. Only I'm not sure which method is right. Let's find out.

Wrt to sRGB -> XYZ matrix: There is no such thing as an "official DCI matrix". The matrix you'll want to use depends entirely on your source's colorspace.

btw: my name is not Tina. It's the name of my daughter's piggy puppet. And my reduser name, apparently. Don't ask

Wolfgang Woehl

 

[fabrizio carraro]

Wolfgang, trying degamma 2.6, rgb->xyz conversion (sRGB matrix) and regamma 2.6 I visually got a result almost identical to the one obtained by easyDCP. Now to complete a completely open-source DCP package there are two issues missing:
1) a quicker way to convert to jpeg2000 (on my Macintosh 8 core easyDCP encodes fast at about 10 frames/sec, while openJpeg runs at 5 secs/frame on one processor, meaning that cutting the script in slices it would achieve about 0.7 seconds/frame, which is still quite a long time)
2) how to encrypt it? It would be enough to generate the local private/public keys (easy to do with openssl) then encrypt the MXFs (how???), and add the extra info into the XML files (how???), get a player's certificate and generate a key for it (how??)
Did somebody manage to to that? I have no idea where to look for info about that.

Fabrizio

 

[fabrizio carraro]

ooops, i meant degamma 2.4 (NOT 2.6), rgb to zyx and regamma 2.6.

 

[Wolfgang Woehl]

Fabrizio,
gamma of sRGB is ~ 2.2 (approximately, see http://www.brucelindbloom.com/WorkingSpaceInfo.html#Specifications, the note on sRGB gamma). While relatively small the difference to gamma decompression from 2.4 is noticeable.

Are you adapting to DCI Reference White? If not this might explain the difference to other packages.

Now the "knowledge" about this is all over the place. Some suggest to target D50 (no idea why), After Effects offers 5900K through its color management preset "DCDM X'Y'Z' (gamma 2.6) 5900K (by Adobe)" and the DCI specs say x=0.314, y=0.351 (~ 6300K).

I've written an ad hoc script to play with the numbers, see http://ruby.pastebin.com/fe99b725

Wolfgang Woehl

 

[Wolfgang Woehl]

And, yeah, having image_to_j2k churn through an hour worth of tiff's for days feels more like a workcrawl rather than a practical workflow. CUDA to the rescue. One, kind-of-working, example is CUJ2K (http://cuj2k.sourceforge.net/index.html)

 

[M Most]

2) how to encrypt it?

Why do you need to encrypt it?

 

[Pavel Klepinin]

I've finally done windows script (.bat) for video->dcp convertion.
This script is based on some Wolfgang Woehl's researches, other post of this thread and a bit of my own experiments.

I've used some additional programs, such as:

MPlayer
ImageMagick
Open Cinema Tools

So.

Script itself (%catalog%/script/repack.bat) as well as catalog structure are in the attachment.

Some important moments.

This script is made for 3d dcp, 1 channel sound, for output resolution of 2048x853. But you can easily modify it for your own workflow.

How to use it.

Edit %catalog%/script/repack.bat.

Set var DIRNAME as your repack directory name (it's %catalog%)
(Exc.
rem Project Directory Name
SET DIRNAME=E:\Temp\3dContent\007-Sign)

All directory names - without ending "\".


Put your audio file into %catalog%\input\audio\
Format - Wav
Depth - 24 bits
Channals - Mono
Freq - 48 kHz

Its name is given for var AUDIOINPUT
(Exc.
rem AudioFile Name
SET AUDIOINPUT=sound.wav)

Put 2 MPlayer-convertabe (uncomressed will be ok) video files for right and left eye in %catalog%\input\video\
Video will be resized to 2048x853 (edit 2 lines such as %MPLAYERDIRNAME%\mplayer -vo pnm -zoom -x 2048 -y 853 "%DIRNAME%\input\video\%VIDEOINPUTLEFT%" if you need other resolution)

Set names for vars VIDEOINPUTLEFT and VIDEOINPUTRIGHT
(Exc.
rem Video for Left Eye
SET VIDEOINPUTLEFT=Left_Eye_Render.avi
rem Video for Right Eye
SET VIDEOINPUTRIGHT=Right_Eye_Render.avi)

Set directory names for 3rd part programs
(Exc.
rem MPlayer Directory Name
SET MPLAYERDIRNAME=D:\Pak\Programs\MPlayer
rem ImageMagick Directory Name
SET IMDIRNAME=D:\Pak\Programs\ImageMagick-6.5.8-4
rem Open Cinema Tools Directory Name
SET OCTDIRNAME=D:\Pak\Programs\opencinematools)


Set other vars according to your needs
(PROJNAME - Any String - Name of your project
CONKIND - I use "test", I've used one other, but it didn't work. Need more experiments.
ANNTEXT - string - annotattion text
and so on).

Run %catalog%/script/repack.bat

Files in the %catalog%/out/ directory will be your dcp project

If everything is ok you can delete %catalog%/processing/ directory with temp files

P.S.
Any questions?

On my machine WinXP Core2Duo 2.8 GHz, 2048 Gb, 17-seconds' clip repacked in about 1 hour.

 

[Wolfgang Woehl]

Hi Pavel,
your convert line doesn't provide for gamma decompression (before sRGB to XYZ) and gamma compression (after colorspace conversion) to 2.6. Is that on purpose?

Also order of parameters matters with "convert". So padding with zeroes after the "-recolor sRGB to XYZ" gives you a different result than doing so before. Have you tested the difference? I'd put the "-depth 12" before any color transformations. But then it never occurred to me to try the other way around, so there.

Right now I'm trying to wrap my mind around the "White" issue, stuff like mastering white, calibration white, white gamut, chromatic adaptation. Read an interesting paper: "SMPTE Digital Cinema White Gamut Practices Study Group Report" (SMPTE Journal, October 2009) and how a DCDM/DCP mastering process would acknowledge the problems identified by the study group and adjust.

I've realized (testing and reading the paper) that transforming to DCI calibration white (0.314, 0.351) is not right and I'm kind of close to "realizing" that doing no chromatic adaptation at all (the DCI specs suggest that the luminance headroom can accomodate a number of standard illuminants, like D65, D55) is not the right way to see your mastering whites in projection either. Have you done any testing in relation to this?

Btw: http://www.digitalcinemanamingconvention.com/ lists, well, naming conventions. Useful.

 

[Pavel Klepinin]

Thnx 2 Wolfgang, fixing errors and moving on.

your convert line doesn't provide for gamma decompression (before sRGB to XYZ) and gamma compression (after colorspace conversion) to 2.6. Is that on purpose?

Nope. It's nothing, but my mistake. Looks like i've used your old script line

for f in *tif; do
convert $f -alpha Off -resize 2048x1080 -recolor "
0.4124564 0.3575761 0.1804375
0.2126729 0.7151522 0.0721750
0.0193339 0.1191920 0.9503041" -depth 12 ../tiffs-xyz-scaled/$f;

Instead of

for f in *tif; do
convert $f 
-alpha Off -resize 2048x1080 -depth 12 -gamma 1.181818 
-recolor
"0.4124564 0.3575761 0.1804375
0.2126729 0.7151522 0.0721750
0.0193339 0.1191920 0.9503041"
../tiffs-final/$f;

Is gamma compression/decompression can be replaced by adding "-gamma 1.181818" or it's some other thing?

Also order of parameters matters with "convert". So padding with zeroes after the "-recolor sRGB to XYZ" gives you a different result than doing so before. Have you tested the difference? I'd put the "-depth 12" before any color transformations. But then it never occurred to me to try the other way around, so there.

Really? I didn't know that. I thought it was just a set of parameters order-independent. So i can fix it shortly.
But as for testing the difference.
Here's a trouble. How are you comparing images. How are you playing dcp on computer. Demo version of easyDCP (which i use) shows the wrong colors (as if there is no need in colorspace conversion). So, i compare colors by playing dcp on Dolby DSS-200 server with original image/video on my notebook. It's not the right measurement. I think it should be some other way. What programs/tools are you using?

And the question about all this white staff. Is this white-correction depends on server/screen/video type and so on, or it's universal and finding the magic numbers will bring some kind of magic workflow?

 

[Wolfgang Woehl]

Is gamma compression/decompression can be replaced by adding "-gamma 1.181818" or it's some other thing?

Colorspace conversion has to be done with linear light numbers. Assuming your source material is sRGB (with a standard gamma of approximately 2.2) you would have to

convert srgb.tiff -depth 12 -gamma 0.454545 -recolor "$srgb_to_xyz_matrix" -gamma 2.6

1 / 2.2 = ~ 0.454545, so applying this value with convert you "decode"/"decompress" the 2.2 gamma and get linear color values.

As a final step -- /after/ any other operation (colorspace conversion, maybe chromatic adaptation) -- you apply the DCI spec'd gamma of 2.6.

How are you comparing images. How are you playing dcp on computer. Demo version of easyDCP (which i use) shows the wrong colors (as if there is no need in colorspace conversion). So, i compare colors by playing dcp on Dolby DSS-200 server with original image/video on my notebook. It's not the right measurement. I think it should be some other way. What programs/tools are you using?

Like you say: It's not correct but if you don't have a calibrated display you're pretty much left with subjective judgement.

And the question about all this white staff. Is this white-correction depends on server/screen/video type and so on, or it's universal and finding the magic numbers will bring some kind of magic workflow?

Here's a quote from the study group report:

"Regarding a specific white point for digital cinema, there is wide-spread disagreement across the production, exhibition and manufacturing communities."

From the data and proposals in the report I take it that targeting DCI's calibration white (0.314, 0.351) is not the right thing to do. But the stuff about white gamut and their proposed virtual white hasn't sunk in enough yet to know the numbers for a chromatic adaptation from sRGB's D65 to DCI projection.
It has been suggested that DCI spec'd projectors accomodate a range of source white points but fact is that a white out of mastering isn't necessarily reproduced at the projector's end.

It'd be great if M Most could chime in and share his experience and practices, much appreciated (Would also be great to learn more about DCP content protection and encryption, of course. Mike, your question seemed to suggest the approach was twisted?)

 

[Pavel Klepinin]

Remade script according to Wolfgang's remarks.

And, yeah, having image_to_j2k churn through an hour worth of tiff's for days feels more like a workcrawl rather than a practical workflow. CUDA to the rescue. One, kind-of-working, example is CUJ2K (http://cuj2k.sourceforge.net/index.html)

As I understood, CUJ2K makes dcp-incompatible j2k files, they are needed of further transformation, that minimizes the time profit, am I right?

 

[Wolfgang Woehl]

Pavel,
apparently the problem is related to markers in cuj2k's output (QCD). I've contacted the authors (student project at University of Stuttgart) to see what can be done. Anyway, the project demonstrates that an acceleration with commodity hardware (GPUs) is feasible. Something will come up.

Another option would be dedicated boards like the intoPIXs (see http://www.intopix.com/products/index/index/id/17/lang/en). I'm looking into prices etc.

 

[Dan Hammond]

DCP creation with CineAsset from Doremi Cinema

DCP creation with CineAsset from Doremi Cinema

I thought you may find interest in a windows based software solution to create DCP's from Doremi Cinema called CineAsset. A free trial download is available at http://www.doremicinema.com/cineasset.html

 

[Chris Perry]

DCP creation with OpenCinemaTools

DCP creation with OpenCinemaTools

Hi all. This thread has been immensely helpful and fascinating.

Yesterday I tried creating a Linux command-line script to execute the entire pipeline. I used Pavel's bat file as a model. We had to compile (and re-compile) lots of things (openjpeg, asdcplib, etc) but eventually I got a pkl.xml file out the other side.

I have a few questions/comments:

1) audio. It seems that asdcp-test is used to create the audio.mxf file from single-channel 24bit WAV files. There don't appear to be command-line args to identify the number of audio channels or their location (aka L, R, sub, etc). How do we keep full control over the audio during this process, especially when we have stereo and/or 5.1 surround sound?

2) Previewing the final DCP. This has come up a few times on the thread. The corrected colors look strange on my computer LCD, of course. Is the only way to see how they ultimately look to take the DCP to a theater?

3) More on colors. If my source material was finaled on a Mac, do I need to use a different gamma/color correction than the one presented in this thread?

4) Time. Converting to TIFFs and then to jpegs does take time. However, once the images have been extracted from the original movie file itself, the process is easily run in parallel. We're planning to distribute the conversion to a cluster of linux boxes, hopefully that will help. If we create any tools that facilitate this, we'll certainly share them.

Thanks in advance for your input.

- chris

 

[Pavel Klepinin]

1) audio. It seems that asdcp-test is used to create the audio.mxf file from single-channel 24bit WAV files. There don't appear to be command-line args to identify the number of audio channels or their location (aka L, R, sub, etc). How do we keep full control over the audio during this process, especially when we have stereo and/or 5.1 surround sound?

As i understood, asdcp-test is made for multichannel audio (5.1 i think). You feed asdcp-test by several wav single-channel files. Each for L, R, sub and etc. And it generates real multichannel audio.
So, if you need to make stereo audio, you should make in your audio editing program 2->5.1 conversion. And render 6 single-channel wav-files. These 6 files appear to be command-line args for asdcp-test.

2) Previewing the final DCP. This has come up a few times on the thread. The corrected colors look strange on my computer LCD, of course. Is the only way to see how they ultimately look to take the DCP to a theater?

It's my big question too. I use easyDCP player, but demo version shows wrong colors. I don't know how to check it on computer.