Nice Shoes R3D form

[shashbugu]

Thanks for all the interest in R3Dform!

If you have any questions please feel free to contact our support team at r3dform_support@niceshoes.com

Just put more information on your site.
What?
why?
how?
demo?
workflow?
tutorial?

If you are going to charge a thousand bucks for a conversion tool, don't be secretive about what you are selling.

 

[Stefan Scherperel]

I'm sorry, but like others have said, I shoot on the red so that I can have a raw file. If I wanted to end up with baked in results in my clips I'd shoot with another camera. With the way adobe works with RED cineX and all the other RAW capable products out there, why would I want to convert all my footage?

 

[Rainer Fritz]

I'm sorry, but like others have said, I shoot on the red so that I can have a raw file. If I wanted to end up with baked in results in my clips I'd shoot with another camera. With the way adobe works with RED cineX and all the other RAW capable products out there, why would I want to convert all my footage?

for offline editorial..... ?? daylies/rushes ??

 

[Simon Blackledge]

I'm sorry, but like others have said, I shoot on the red so that I can have a raw file. If I wanted to end up with baked in results in my clips I'd shoot with another camera. With the way adobe works with RED cineX and all the other RAW capable products out there, why would I want to convert all my footage?

rushes/dailies
offline
vfx

To name a few. Both have there uses.

s

 

[Stefan Scherperel]

Premiere CS5 - No need for Offline.
Rushes and Daylies, all done quickly with CS5.

 

[JustinP]

For me, when working on low budget productions off a laptop I would love to see an app that just gave me reliable realtime playback at 1/4 rez or higher for the full length of a clip or edit (currently it just keeps up at 1/8th on 4K material). Many productions like to do transcodes for the offline just for performance whilst editing especially when people are often editing in FCP on a laptop. I fully understand and agree with the idea of working with native or the now available (YAY!!) trimmed R3D's when finishing/doing final grades etc.

Hi Quentin,

I think this is an example of the perfect situation in which to use R3Dform.

Currently, R3Dform supports full and half res -- are you interested in seeing it handle 1/4 or 1/8 res in the next version? That would make it possible to go faster than real time with just a single laptop!

 

[Dan Kanes]

Premiere CS5 - No need for Offline.
Rushes and Daylies, all done quickly with CS5.

Really?

I'd love a tutorial on creating dailies in realtime or faster outputted to ProRes 4:2:2 using cs5.

Frankly, I'd be happy with a tutorial on doing this same thing using r3dform.

Perhaps there is a way to do it without resorting to the use of compressor?

Is there a way to do this?

 

[JustinP]

Okay - this program seems like it's on the right path - debayering r3d's in real time without a red rocket card.

My issue with R3Dform at this time is its dependence on compressor.

I hate the compressor interface.

Hi Dan,

Thanks for your interest in R3Dform!

Right now, R3Dform is just compatible with Compressor because that program is a cost-effective solution, is available to anyone with Final Cut Pro and can create any deliverable you would need. What other program would you prefer R3Dform to be compatible with? Episode Engine? Sorenson Squeeze?

 

[Dan Kanes]

Hi Dan,

Thanks for your interest in R3Dform!

Right now, R3Dform is just compatible with Compressor because that program is a cost-effective solution, is available to anyone with Final Cut Pro and can create any deliverable you would need. What other program would you prefer R3Dform to be compatible with? Episode Engine? Sorenson Squeeze?

I'd like it to have its own engine for transcoding like Rarevisions 5DtoRGB does.

Otherwise a step by step tutorial to using r3dform with compressor on youtube would be sufficient. Does it mux the audio stream in? Are RMD looks applied to each file? Thanks!

 

[Gunleik Groven]

Yep, STAY RAW! (until the final target)

Actually, no... - Even though I'd like not to have to make DPXs or whatever, it is important that the software works right...

That is that it decodes the RAW into RGB and that you don't touch the RAW again further down the grade.

That hasn't always been the case with RAW graders, or it has been rather tricky to know where in the pipeline you leave RAW ang go into an RGB grade.

Thus the thing I like about DPXs etc is that they are really quite foolproof - if developed right.

Just a thought, not really related to NiceShoes, though...

 

[JustinP]

I'd like it to have its own engine for transcoding like Rarevisions 5DtoRGB does.

Otherwise a step by step tutorial to using r3dform with compressor on youtube would be sufficient. Does it mux the audio stream in? Are RMD looks applied to each file? Thanks!

The YouTube idea is a good suggestion and we're working on putting something together ASAP.

To answer your other questions, R3Dform does not mux audio and it does not use RMD looks. You can affect the gamma and the color in the settings (or any combo of the two) but R3Dform was designed just to debayer and to convert the R3D files, leaving the creative decisions to the artists.

That's the short answer as to why R3Dform doesn't support LUTs, scaling, transforming, or any other creative decision.

 

[JustinP]

We must get more technical info on this product! What is the method by which the software is debayering / transcoding the R3Ds into QuickTimes? How do the results look? Are the renders actually useable for finishing, or just this strictly for generating 'offline' files?

This was developed by Nice Shoes -- a professional finishing company -- so the results are certainly usable for finishing.

I'm wondering how the conversion is happening faster than real time without an accelerator card. To my understanding, the SDK simply provides an RGB framebuffer, which would make everyone's access speed exactly the same.

Did they gain access to the raw data and are using openCL or CUDA or something? Doesn't seem like they say so, and they specifically say no expensive hardware. Hmm, maybe I'm misreading what they are saying, but I thought it said faster than real-time. Only way I can figure that is if it's not full res?

R3Dform works as fast as your CPU and your write I/O work by implementing parallel processing and high performance computing methodology.

R3Dform works like a render farm to take advantage of all your CPUs and all possible threads. As an example, we used three 8 core first generation Apple machines to convert 4k R3D files to 2k DPX files at 775 mb/s -- that’s 2.5x real time.

It is recommended to set the number of r3d cores to 1.5x the amount of cores you have in your machine for maximum power. To be more collaborative, you should set that number lower – managing the number of cores page 15.

As for their speed claims, I'm going to either say they're misleading about the number of systems used for their examples or it's outright BS. For starters, even if you can pull off a software debayer at 100X real-time and feed 4K frames right into the ProRes codec, you'll be lucky to get better than about 75% of real-time on the encode because ProRes is a huge bottleneck. Other than that, proper multithreading on a good 8 or even 6 core system can give you real-time debayer at full resolution for a 4K R3D source. But you still need proper LUT application, transform and scaling, which is where a good GPU would be handy. Same principal of how Premiere CS5 and now a few others are chewing through multiple streams of REDCODE in real-time or even a single stream at full quality on a decent system.

So given that, I don't doubt the application can grind through the REDCODE if properly coded, I just question what they do with it after the fact.

The advantage to having a RED Rocket is that it handles the debayer processing, LUTs and transformations/scaling and keeps the rest of the system free to do other things. We will see its advantage more and more as applications on the market increase their support for the Rocket.

The speed issue was addressed above.

Regarding, the other half of this question, R3Dform allows you to preset gamma and color space into a profile so it can be recalled at anytime. We feel strongly that LUTs should be set as part of the creative process as should any other creative decision such as transformation, scaling, etc.

Additionally, these decisions are specific to a single user or a single application and are only applicable to the current machine you are working on. The debayering process, however, is universal and works across multiple platforms. This is important to note when sharing a cut with an editorial shop, a color grading suite, a VFX house, etc.

Here's a related selection from the R3Dform manual:

Managing Render Job Profile
A render job profile is the definition of the output files of a render job. Basically the definition consists of two types of data - the image processing settings and the naming patterns.

You can create new job profiles, edit, delete and overwrite existing profiles.

Image Processing Settings
The image processing settings that you can specify for the output files are:

Frame size — the size of the output frame. The possible values are half and full frame (according
to the original movie frame size).

DPX bit depth — the bit depth of resulting DPX file. The possible values are: 10 and 16 bit.

Gamma curve — the gamma curve used when decoding frames from the source movie. The possible values are:
• REDLOG
• Rec709
• Linear
• REDspace
• PDlog685
• Pdlog985.

Color space — the color space used when decoding frames. The possible values are:
• CameraRGB
• REDspace
• Rec709.

Decode quality — the quality setting for decoding. There are two quality settings: Good and
Premium.

Note: If the frame size is set to full, this setting is ignored and the conversion always uses premium quality.

Download the full manual here.

 

[Thomas Worth]

I'm sorry, but like others have said, I shoot on the red so that I can have a raw file. If I wanted to end up with baked in results in my clips I'd shoot with another camera. With the way adobe works with RED cineX and all the other RAW capable products out there, why would I want to convert all my footage?

The idea is to transcode to a format that can represent pixels discretely without having to "rebuild" data on the fly. Even 4:2:2 formats are guilty of this. Subsampled formats alone prove that you cannot be 100% sure you're not getting robbed of quality. Realtime debayering only exacerbates this. Only discrete RGB formats are 100% reliable in this regard.

The idea of Cineon/DPX was to be able to get as close as possible to storing the full range of information the 35mm negative was capturing. This would still allow full control over color at a later time, without having to "bake" decisions into the footage. The same should be possible with RED (as it is with still cameras), although I don't know enough about the sensor to know exactly how much dynamic range you would need to store everything (I bet a 16 bit linear format would do it).

The problem with raw formats is you need special software to get it to work, and each implementation may differ slightly. If it's just discrete pixel values, there is no ambiguity as to what the data should be because you have each and every pixel accounted for. This, in my opinion, is a more robust method of working with the data albeit at the expense of storage space and throughput. It also means you need less CPU power.

There's nothing wrong with shooting RAW. It's a perfectly adequate acquisition scheme, considering the challenge of deducing RGB from a Bayer-filtered sensor. So is H.264, or DVCPRO HD, or DV. But working with uncompressed data is without compromise (yes, even compared to RAW, as long as you have enough bits), and just works, all the time, on any machine, without question and is 100% future proof.

The "do I really need to do this" issue comes up all the time with 5DtoRGB. The answer is "nope." But if you're obsessed, as I am, with utmost quality, then uncompressed RGB is the only option you can truly count on.

 

[sebastian toro]

Regarding, the other half of this question, R3Dform allows you to preset gamma and color space into a profile so it can be recalled at anytime. We feel strongly that LUTs should be set as part of the creative process as should any other creative decision such as transformation, scaling, etc.

at least you need to take care about asa setting for technical debayer. if you cant manage that and you are going 10 bits you could be crippling data and this is not a artistic decision.

 

[Dan Hudgins]

DPX, 10bits, Cineon, Grain, tone banding, grading step placement

DPX, 10bits, Cineon, Grain, tone banding, grading step placement

The idea of Cineon/DPX was to be able to get as close as possible to storing the full range of information the 35mm negative was capturing. This would still allow full control over color at a later time, without having to "bake" decisions into the footage. The same should be possible with RED (as it is with still cameras), although I don't know enough about the sensor to know exactly how much dynamic range you would need to store everything (I bet a 16 bit linear format would do it).

The original 10bit Cineon files from film scans only used the range of 95-685 out of 0-1023, with hard clip at 685 most of the time, but with an option for softclip. The overrange density values were so that scanner did not have to compensate for exposure errors in the negative, rather than so much for some "head-room", base density was fixed as was d-Max, the idea was to slide the clip points up and down on the data, but you do not grade digital cinema footage that way since it is linear with a hard clip at the top and not on an S-curve like film. Cineon requires conversion from its gamma 1.7 to display gamma 2.2, and that introduces an expansion in the lower tones that makes the tone banding on monitors closer to an 8 bit image, in the film recorder I guess the gamma 1.7 images went through a LUT that was desigined to reduse the scaling errors. That brings up the issue of how the bits are scaled in making the compressed files for DCI playback. The type of heavy grading people do today with heavy 3D-LUTs was not in Kodak's plan for Cineon/DPX type data from filmscans, it was more like a timed 2nd generation film print in its scope of tonal adjustment, what could be done with a Bell and Howell dichroic light house.

If you made a COPY of the DPX file that was graded the copy was to be hard clipped at 685 so there was not "extra" data above that, at least if you read the way Kodak says the softclip should be done only ONCE in the image processing.

When you process digital cinema camera images into DPX you cannot put the highlights above the white clip point as you would with some film scans since you will lose the highlights making it look more like a video transfer such as REC709 without softclip.

Since the digital cinema white clip must match the DPX white clip there is no extra range, at least I would hope people would not be lopping of the top of the Digital Cinema Camera's tonal values.

What I have been doing is to output to TIFF 48bpp with the Digital Cinema Camera's white clip at 65535 or a tad under that, that way you have the full grading range in a RGB file, and you do not need to conform the end use from the un-deBayered data, which has issues in setting the de-Bayer values going back.

If you make an ungraded iso/k adjusted de-Bayer RGB 48bpp gamma 2.2 uncompressed as the master then you have someting "stable" to look at and downgrade from.

In the de-Bayer program I am working on I have options to re-generate after editing from the RAW files, but that is within my workflow, getting everything to work with RAW leads to Interoperability issues that working with a DPX or TIFF don't have because of their wide support and long standing use.

The reason people think 10bit DPX worked well with film, was that the high grain in the scans worked as dithering. Whenever you have grain in a high resolution image it acts to reduce the tone banding like using an extra bit or two. For images made by Digital Cinema cameras that have been de-grained in the de-Bayer process there may be less grain to dither the limited tonal range in 10 bit DPX files, so 16bit DPX or TIFF 48bpp files should be used for RGB images that will have heavy grading applied, that is the RGB file is a long scale Intermediate step between the RAW data and the finished result format. By skipping the Intermediate RGB step bewteen RAW and compressed finish formats, where is the fine/finish grading done, not in the compressed finish format I hope?

 

[Thomas Worth]

When you process digital cinema camera images into DPX you cannot put the highlights above the white clip point as you would with some film scans since you will lose the highlights making it look more like a video transfer such as REC709 without softclip.

That's assuming you write the DPX sequences the same way you would film sequences. You don't have to do this, though. Most programs seem to expect 10 bit DPX, but the spec provides for 16 bits per channel. Also, data can be flagged as linear, making better use of the data for a linear workflow so you don't throw away data when converting log to lin as you pointed out.

What I have been doing is to output to TIFF 48bpp with the Digital Cinema Camera's white clip at 65535 or a tad under that, that way you have the full grading range in a RGB file, and you do not need to conform the end use from the un-deBayered data, which has issues in setting the de-Bayer values going back.

You can technically do the same thing with DPX, but I'm not sure how much support there is for DPX files written this way. Frankly, I think the DPX format is simpler than TIFF, and the headers and fields are better suited for motion picture work (when they're supported by the app).

getting everything to work with RAW leads to Interoperability issues that working with a DPX or TIFF don't have because of their wide support and long standing use.

And EXR for HDR (greater than 16 bit) applications.

For images made by Digital Cinema cameras that have been de-grained in the de-Bayer process there may be less grain to dither the limited tonal range in 10 bit DPX files, so 16bit DPX or TIFF 48bpp files should be used for RGB images that will have heavy grading applied

I tend to agree, at least until 16 bit DPX support becomes more prevalent.

 

[Dan Hudgins]

"Log" vs. Lin

"Log" vs. Lin

And EXR for HDR (greater than 16 bit) applications.

Log as far as I can tell from the Kodak Cineon specs is gamma 1.7 with the black level lifted to a value that depends on how much over-exposed the camera negative was, the base density being put under black clip for some over-exposed shots, but it depends on how the scanner is setup.

Gamma 1.7 will give less scaling errors than "linear" gamma 1.0 data for a given number of bits since the gamma needs to be above 1.0 for normal viewing, like gamma 2.2 on a monitor.

With the same Cineon file used from the scanner in the film recorder without being copied and adjusted in grading, the conversion from gamma 1.7 to print density was done with some kind of LUT to compensate for the gamma 2.4 of the CRT in the film recorder, so there had to be some scaling errors in the film recorder, but if it had a 10bit DAC in its video board those would be under the grain of the dupe-negative stock used, probably.

The reason I said that using 16bit 48bpp TIF files at gamma 2.2 is preferred is that with that you don't need anything more for higher bit Digital Cinema Cameras like 18bits linear HDR, because once you lift the 18bit data from linear color space to gamma 2.2 for display on a monitor you don't even need 16bits maybe 15 might be enough. So unless you are doing something very odd, just about any HDR image's RAW data can be converted into a 48bpp TIF file and hold the shadow detail if the data is on a gamma 2.2 curve, which lifts the shadow detail up more than the so called "Log" curve that is only gamma 1.7.

I don't think you need EXR for images made in a Digital Cinema Camera, for computer graphics maybe they are useful in special ways, but for what noise and banding you can see, a gamma 2.2 48bpp TIFF file shoud be able to hold what a camera can photograph and your eyes can see after that is graded down to the range the monitor/projector can show.

There is an Enormous improvement going from 10bit Cineon/DPX LOG to 16bit 48bpp TIFF at gamma 2.2 with regard to the shadow detail it can hold, more so than going from 10bit "Log" to 16bit Lin, because the dark tones are streched out, you would need to start with maybe about 19bits Lin to lose much at 16bits gamma 2.2.

==

About the software that is the topic of this thread, would it be possable to speed up the conversion of R3D files to lesser compressed formats that also use JPG2000 type wavelet compression, without doing any formal de-Bayer at all, just some kind of scale on the wavelet control values?

You could not grade that way becasue the wavelets are the brightness data, but you might be able to do a very fast conversion by scaling the wavelet control values?

 

[Mel Matsuoka]

The problem with raw formats is you need special software to get it to work, and each implementation may differ slightly. If it's just discrete pixel values, there is no ambiguity as to what the data should be because you have each and every pixel accounted for. This, in my opinion, is a more robust method of working with the data albeit at the expense of storage space and throughput. It also means you need less CPU power.

There's nothing wrong with shooting RAW. It's a perfectly adequate acquisition scheme, considering the challenge of deducing RGB from a Bayer-filtered sensor. So is H.264, or DVCPRO HD, or DV. But working with uncompressed data is without compromise (yes, even compared to RAW, as long as you have enough bits), and just works, all the time, on any machine, without question and is 100% future proof.

Thank you Thomas, for stating what I'm surprised nobody has really said here yet.

It all matters what your specific workflow is, of course, but I can tell you as a shop based on an FCP, Apple Color, Shake and After Effects based, TVC-oriented workflow, we've mostly abandoned working directly with the raw R3Ds unless we absolutely need to.

The performance issues with working directly on the raw are significant, but at least still somewhat tolerable in Color, but it pales in comparison to the horror of opening up an old project a year later, only to find that all your grades are fucked up because the color science has drastically changed in that short year, and the application has no idea that it should be using a specific legacy implementation of the color science.

This happened to me on a 10-spot series (with dozens and dozens of edits) of national ads I worked on, pre-build20 color science. When the client came back a year later, wanting to edit and grade revised versions with alternate takes, I nearly jumped out the window when I realized that none of my original grades worked with the new version of the Color plugin and RED QT codec.

Yes, I could have reverted to the older setup (which I did, much to my annoyance), but this was a hard lesson for me just how much of a moving target RED post-production is. In the end, I felt that doing the original grades in raw wasn't really worth the time and headaches of trying to figure out why everything unraveled a year later.

Working with well-developed transcodes of well-shot footage is always faster and fool/future-proof. I don't have to explain to other RED-unsavvy AE artists in the facility why the footage in their comps looks different after a color science update or why they should now be using REDcolor--and now, REDcolor2--instead of REC709 (getting them to understand how to properly use AE's own color management is hard enough!). And working with transcodes also allows me to kick the damn client out of my suite faster, because I don't have to wait seven-thousand years for everything to render.

Don't get me wrong, I absolutely love the fact that RED color science is constantly improving, but I feel that overzealously diving in headfirst into an all-raw workflow is too dangerous for a lot of real world situations, such as a facility based TVC pipeline, where you don't have time to be nerding about REDuser trying to figure out why things aren't working properly, and then conveying that knowledge to the other artists in the facility (who should be spending their time doing what they're best at, instead of having to become an ad hoc DIT every time a new firmware build and/or color science update is released)

Simply exclaiming that "TRANSCODES ARE SO OLD SCHOOL!!!" is to ignore the reality of many established mission-critical workflows, and further perpetuates the stereotype of many RED proponents as quasi-religious fanatics. Please cease and desist

 

[Nick Shaw]

…the horror of opening up an old project a year later, only to find that all your grades are all fucked up because the color science has drastically changed in that short year, and the application has no idea that it should be using a specific legacy implementation of the color science.

RED have partly fixed that with the recent SDKs which allow you to choose which colour science version is used. However that selection is not currently available in Color.

I would argue though that they have not gone far enough, because there is only the option to use colour science versions 1 or 2. What about version 0 – the original colour science? Although that is now considered depreciated because it has been improved on, I think it should still be available for compatibility when opening older projects.

 

[Gunleik Groven]

As I mentioned earlier, there's been too many drawbacks with the itirations I have seen in pure RAW workflows for me to really recomend them for larger productions.

That is not to say that we won't get there or that it isn't useable. But the beauty of DPX's - once transcoded right - is that you or the colorist don't have to worry about some odd metadataglitch left in there from the shoot or an overzelous prod-ass somewhere in the signal-path.
BTW: I haven't - like Mark - tested them all, but the way it is implemented in Color f.ex. has matured, but is far from optimum.

I am looking foreward to have a look at the DaVinci implementation, though and I am sure it will be right (I don't mean to be the provocateur here, I am sure a lot of people have it working very good for many situations) it's just a bit geeky still.

Seeing the redlogFilm thingy, makes batch much less risky and has lifted the "worstcase" scenarios on batch transcode from "really bad" to "pretty darned good".

Which is good, and a bit more foolproof and will make for more standardization in RED workflows, RAW or not. It's easy to explain, and it won't give really really bad results, as I have often seen.

A bit besides the discussion here, but really not:
Untill things stabilize (which probably means: never), there will be huge use for good transcoders for foolproof workflows for a lot of project-scenarios.

Why I shouldn't just use the rocket, is a thing I don't understand, though....