Opinion: Why Redcode is the future of RED?

[David Battistella]

That sounds about right. Open CL has a lot of potential, but there are a lot of tasks that a GPU isn't suited for... I guess this is one of them!

For example, REDCODE needs major horsepower to debayer on the fly and show them to you in QuickTime. I'm pretty sure if they could take advantage of further resources they would.

I think I asked Graeme about this a while back and it might have something to do with the wavelet compression.

David

 

[Justin O'Neill]

So I would shoot R3D files copy them to my system and edit in realtime in FCS, Premiere. Think of it as the high end DV codec of the future, offering RAW in Realtime would widen the use of the RED system.
David

David, I must be missing something because I can do what you are describing on a medium powered PC or a fast laptop right now.

In REDCINE-X, Premiere CS4, and Vegas 9 you can drag n' drop REDCODE RAW files into the timeline and edit the RAW files directly in realtime. RED's SDK makes this possible with realtime resolution switches from full to 1/8.

On my medium powered Vista PC and Premiere CS4 I can edit 4k REDCODE RAW realtime at 1/4 resolution. I can switch to full res and back in realtime with a click of the mouse. I also have access to REDCODE's RAW sliders with a click of the mouse. REDCINE-X which is a FREE program offers these exact options as well.

This is the state of REDCODE RAW post workflow as of right now. How is this different then what you are asking for?

 

[David Battistella]

David, I must be missing something because I can do what you are describing on a medium powered PC or a fast laptop right now.

In REDCINE-X, Premiere CS4, and Vegas 9 you can drag n' drop REDCODE RAW files into the timeline and edit the RAW files directly in realtime. RED's SDK makes this possible with realtime resolution switches from full to 1/8.

On my medium powered Vista PC and Premiere CS4 I can edit 4k REDCODE RAW realtime at 1/4 resolution. I can switch to full res and back in realtime with a click of the mouse. I also have access to REDCODE's RAW sliders with a click of the mouse. REDCINE-X which is a FREE program offers these exact options as well.

This is the state of REDCODE RAW post workflow as of right now. How is this different then what you are asking for?

No seeing FULL framerates and FULL resolutions on the MAC, maybe an M proxy with a lot of RT settings to draft, etc, but not in full resolution or frame rates.

Still much more reliable to transcode all the media.

David

 

[Craig Parkes]

If the code is not or can not be further optimized then it would be important to make sure that any hardware acceleration (REDROCKET) can interface easily with existing programs (FCS, Premiere, etc.) So if I have a REDROCKET then I can run realtime 2K and 4K editing in Final Cut Studio, for example.

But as it stands right now, I'm not sure this is possible.

David

Assimilate have already released Rocket Fuel, which is Scratch Cine but also utilizes Red Rocket. There is a Red Rocket SDK, which means software makers can ensure their products support Red Rocket.

The speed at which they do so depends on the software manufacturer, difficulty of implementation, and ongoing improvements to the Red Rocket SDK on would imagine.

Seems to me while it won't happen overnight, it will happen.
Final Cut has a number of limitations inherent to the program, rather than REDCODE, that limit the feasibility of realtime 2K and 4K editing in Final Cut (it's RT Extreme engine not historically being designed with those resolutions in mind etc.)

It's not support from RED that is required here, but support from your software manufacturer.

And as is often quoted here - why one earth would you need to edit Realtime 4K on a desktop system? There isn't a deliverable format that exists yet that would require that, there isn't currently any cost effective solution for monitoring in 4K so you can't take advantage of the resolution as you are editing, there are 'standards' when it comes to delivery or color correction in a video colour space for 4K delivery (which is all that Final Cut can natively handle - so you might as well be handing off to Color if that's all you have access to, which means you might as well be using an offline/online workflow) and considering the cost of Final Cut compared to the cost of the hardware you would need to monitor and have a properly color managed output for any other purpose for 4K then you would quickly ask the question why you are using such an underpowered/inappropriate tool for the task at hand. (Nothing against Final Cut, I use it on a daily basis and find it a great tool, but under the hood at the moment it's REALLY not designed for real time editing at 4K, and doesn't scale well when working with non optimized resolutions)

Realtime 1080P editing from a scarlet is obviously a must, and they are aiming for this with the RGB formats that have been mentioned. If these can come to pass on board then you get the most realistic solutions for both problems, those that need simple post workflows but great pictures will have an immediately editable RGB file that pretty much any existing edit system will be able to understand and work with, while those require the BEST pictures will have the option of REDCODE.

 

[Justin O'Neill]

No seeing FULL framerates and FULL resolutions on the MAC, maybe an M proxy with a lot of RT settings to draft, etc, but not in full resolution or frame rates.

Still much more reliable to transcode all the media.

David

So you want full quality, full resolution REDCODE RAW in realtime? I think if RED could do that they would in a second, but the massive amount of data in 4k has to be reduced somehow.

You can definitely get full framerates with REDCODE RAW at lower resolutions. What resolutions can you guys with Macs get with REDCINE-X in realtime?

Sorry if I am misunderstanding you David.

 

[David Battistella]

Assimilate have already released Rocket Fuel, which is Scratch Cine but also utilizes RED Rocket™. There is a RED Rocket™ SDK, which means software makers can ensure their products support RED Rocket™.

The speed at which they do so depends on the software manufacturer, difficulty of implementation, and ongoing improvements to the RED Rocket™ SDK on would imagine.

Seems to me while it won't happen overnight, it will happen.
Final Cut has a number of limitations inherent to the program, rather than REDCODE, that limit the feasibility of realtime 2K and 4K editing in Final Cut (it's RT Extreme engine not historically being designed with those resolutions in mind etc.)

It's not support from RED that is required here, but support from your software manufacturer.

I agree, Apple only responds to what it's users ask for.

As it stands if RED Rocket, can handle playback of r3d in 2K and 4K then why not use that extra HP to edit that in FCP. Not saying I would want to edit 4K just talking possibilities.

In the SCarlet context which I would assume is the widest possible audience then you have to make it a really tight and real time integration with the FCP crowd. I think you'd see a lot of reality tv series and such moving to RED.

David

 

[David Battistella]

So you want full quality, full resolution REDCODE RAW in realtime? I think if RED could do that they would in a second, but the massive amount of data in 4k has to be reduced somehow.

You can definitely get full framerates with REDCODE RAW at lower resolutions. What resolutions can you guys with Macs get with REDCINE-X in realtime?

Sorry if I am misunderstanding you David.

We can see the full resolutions int REDCINE-X, RA and RC, in QT you can see a reference movie in Full, High, Medium and Proxy, but the conventional wisdom is to transcode the r3d to QT media and then edit with that. The on-the-fly debayering is not fast enough (and sometimes quirky) to edit on an OCTOCORE mac. even with fast raids, the redcode can not be decoded fast enough to eliminate an OFFLINE/ONLINE style workflow.

 

[Stefan Christou]

Not going to happen. I remember that Graeme said something like about this, that they'd tried decoding on the GPU early on and it was far less efficient than using the CPU.

Also, as far as I can tell, the reason REDCODE takes so much power to decode is that it not only has to decompress the image but also demosaic it and interpolate the colour channels and then add the gamma curve, white balance and other colour correction. If you wanted a real time REDCODE, it wouldn't be raw anymore, so you'd lose most of its advantages. It'd be better to just transcode to ProRes after shooting it, so you could go back to the original RAW files than doing it on the camera.

That's a shame. If you want realtime on the cheap why not just capture from the HD-SDI?

 

[IAN SUN]

That's a shame. If you want realtime on the cheap why not just capture from the HD-SDI?

Well, I don't think it is a full debayer.

 

[I Bloom]

We can see the full resolutions int REDCINE-X, RA and RC, in QT you can see a reference movie in Full, High, Medium and Proxy, but the conventional wisdom is to transcode the r3d to QT media and then edit with that. The on-the-fly debayering is not fast enough (and sometimes quirky) to edit on an OCTOCORE mac. even with fast raids, the redcode can not be decoded fast enough to eliminate an OFFLINE/ONLINE style workflow.

David,

Since camera raw material is 1/3 the data size of RGB material it seems to me that the bayer pattern is part of the compression so it's not likely that Red will adopt debayering in camera unless it is for a downconverted frame size like 1080p. Likewise debayering algorithms are not all made equal and there is a definite quality gain from taking it a little slower.

What you have to understand is that Apple, Nvidia and AMD are right now in a race to best support the kind of GPU based computation that a real time full debayer requires. You are likely to see this become a possibility on standard hardware much sooner than you think.

IBloom

 

[David Battistella]

What you have to understand is that Apple, Nvidia and AMD are right now in a race to best support the kind of GPU based computation that a real time full debayer requires. You are likely to see this become a possibility on standard hardware much sooner than you think.

IBloom

Thanks for that news Ian.

David

 

[Theo Valich]

Here's a question... what does exactly REDCODE need, as far as computing power go? If possible, I would like to get a straight answer.

Today, a GPU based on 2008-gen hardware [let's take GeForce GTX 280/ Quadro FX 1800/3800/4800/5800 hardware] is capable of handling 128-bit HDR, uses 32-bit float instructions per channel - R:G:B:A. As far as display capabilities go, GPUs are capable of 12-bit per channel output [but true, due to display limitations mostly 10-bit is used]. Oh yeah, 1TFLOPS of compute power.

If we take a look at what kind of hardware is available today from AMD [2009 class: 5870 series - expect professional FirePro cards at NAB 2010], we have a single chip capable of outputting 7680x3200 resolution [25Mpix]. But it is not capable of outputting alone, but creating the whole image using a 3D game engine and apply Anti-aliasing and Aniso-filtering to boot.

Same thing applies to transcoding videos from one resolution to another - GPU still beats CPU, and the only limit is that unfortunate CPU-GPU overhead that is resolved in Windows 7.

What is exactly the problem in writing an application that harnesses that compute capability? It is absolutely true that up until 2008, GPGPU was severely limited [lack of space for instructions etc].

But if somebody could give a set of numbers where the roadblocks are, would be nice - because then a GPU manufacturer could tailor the chip to suit REDCODE more. This is especially true if you look at the fact that nVidia's 2010 cGPU will be capable of booting an operating system without the use of a CPU. Now, if you're able to boot an OS with a GPU, and that same GPU has multiple TFLOPS of compute and output capabilities - where is that roadblock?

Especially since video is essentially a stream operation, and it is much easier for GPU to process the stream data than on the CPU. I might be crazy [have a game development background], but to me, everything is just the data sets, and I don't see REDCODE being a roadblock. Debrief or just the small insight of what's needed where would be most welcome.

As far as my experience with both nVidia and ATI go, both companies work for progress and they want to open new worlds. I didn't had that experience with CPU business.

 

[Kenneth Elkington]

I'm no computer science expert, but there are two potential reasons that i can see that may be hindering the execution of a GPU-based redcode codec. First, GPU stream processors may not be suited to the types of computation necessary for demosaicing. Someone earlier in thread said that Graeme(?) stated this as a problem in the past. The second is that those types of programs are quite difficult to write from scratch as opposed to writing for a purely general purpose CPU, and the technologies designed to facilitate writing these programs, CUDA, OpenCL, DX11, are still relatively new and immature. The first release versions of both OpenCL and DX11 were only released in the past 3 months, and I've yet to see any GPGPU programs use them.

 

[Gunleik Groven]

Having losely followed this thread for a while, I might just repeat stuff, but here's my 2 cents.

I've always thought Redcode to be the most valuable feature of RED. The ability to record and edit far more info than my computers can process in realtime, and deal with the data-overhead in the way I choose - instead of getting a compressed RGB/YUV codec with all the problems baked in.

I know it's weird, but in 1 1/2 years of RED use, the offline/online workflow has become so easy and second nature that I really just have forgotten the past. I can choose - depending on project/client/cash whether I should do a full de-bayer, clip by clip onelight, DPX online for the best result, or a quick and dirty correction in FCP from the Cam Proxies. And then a lot of stages between those two extremes.

If anything, I wish for MORE, not LESS info in the RAW files. They are so flexible and quick to work with that it never stops to amaze me, and I'd much rather be forced to work with S-proxies with more RAW data, than having a crappy F mode in realtime. Computing power, be it GPU or CPU based expands all the time, my files don't. I want the most recorded.

Hey. With the same cam-system and technically spoken: from the same files - I can have a turnaround quicker than P2 and deliver news flash directly from the the M-proxies on the RED DRIVE (yup, I know that's not kosher, but deadlines are deadlines...) AND do a developing pass + a proper online for high-end work.

The flexibility I "lack"/sometimes wish for is higher bitrate, better DR, more info. Not less.

OTOH I have said since the day of the Scarlet announcement, that whitin a year of the relase of the Fixed Scarlet you'll robably find Scarlet support in iMovie and similar apps. Thus i find the "RAW is the new DV" thingy is totally legit.

But rather than crippling the format because it will be used by non-techies (not implying that the OP is a non-techie, but that my mother is... ), I wish RED further develops the path they already are running.

I don't want the camera to decide what kind of production I am doing anymore after using the R1 - and that's 100% tied to the codec philosphy. I want ME to decide the road.

My mother OTOH should be perfectly able to use a new 2/3" fixed scarlet, push the "auto everything" button and happily load her clips into iMovie without even considering what a "developing/technical pass" means or implies, when she takes nice stills and films of my kids. Wth 3k 2/3" fixed lense scarlet that can record directly to 1080, I see that already happening.

So:
Rather than crippling the codec on set, I'd like to see an even more powerfull codec, and better SDK implementations and a better understanding on how signal-chain affects the RED images (like in the case of Color), probably a more mature SDK and RR to be supported by the SDK by default for those that need to do cuts in 4k. -

 

[Axel Mertes]

RAW demosaicing/deBayering via GPU is possible today!

RAW demosaicing/deBayering via GPU is possible today!

Clearly you can do it with GPU power, today.

We use SpeedGradeDI here on our PNY Quadro FX 5800 SDI card.
When feeding Cineform RAW 4K into the system, we can debayer at 4K in realtime via the GPU and apply even grading layers on it.

In earlier days, until including build 15 firmware, the RED RAW codec (based on JPEG 2000, 4 quadrants stored as RGBA, where A is the second G) could be directly read as native RAW, so we were able to deBayer RED RAW in realtime. Other apps had the same ability for a given time.

Unfortunately that hidden door has been closed by RED due to changing and/or encrypting the RAW codec since build 16, for whatever reasons. Right now all apps need to go through the SDK for a full color deBayer prior to processing, which is the bottleneck. Its pretty slow.

RED Rocket can accelerate that on certain systems. But again, you will only get back full color data, which by itself is the bottleneck, as its at least 3 times larger than the RAW. Transferring that much data inside a computer into a graphic subsystem is a problematic task.

If RED would allow to just decode a RAW image from the R3D via the SDK, then we would be able to continue via the GPU approach, like Iridas showed, and which others could follow up on.

I understand that RED wanted to make sure only their own RED RAW demosaic takes place, but I see the option to force all vendors to implement it, but allow them (and their users) to use different demosaic methods as well. That would be the best solution for both side, IMHO.

Comments welcome.

We currently transcode from RED RAW to Cineform RAW or Cineform 444 and then we can continue working realtime.

The RED RAW to Cineform RAW conversion is performed by first decoding the image with the SDK in full resolution, then dropping all interpolate color pixels and recompress the remaining RAW Bayer pattern pixel data using the Cineform RAW compression (which is very similar to REDs internal approach anyway). Clearly there is a loose in recompression, but less that many think. The workflow is a cost effective alternative to all who can't afford e.g. the RED Rocket or other high end gear.


Cheers,
Axel

 

[Gunleik Groven]

Almost had forgotten those pre B-15 days Axel.

The computing overhead exploded with B-15 to the extent that even Scratch wasn't realtime 2k anymore. (at the time).

Have to agree with your take, but - from the looks of it - cineform basically pissed red off with showing a bad debayering (that they quickly rectified) of a codec in constant development. I think the main rationale was to protect REDs (obvious) right to develop the codec further without looking left and right for compatibality.

But the computing overhead added has its distinct disadvantages.

 

[David Battistella]

Gunleik,

I appreciate your comments in this post. I agree with you that on the high end the direction is bang on, but as I would also like the user option to take a faster debayer over transcoding. In a pinch this could be pretty handy and work well in different scenarios even in an offline online scenario.

Like how about using redrocket to send a 1/4 debayer of a QT to FCP. then you cut at 2K resolution, you send to Color where the FULL r3d is read and rendered.

Then you play out the rendered stuff through FCP and a KONA card to whatever you want. this would be a great way to manage all deliverables. This is the very basic workflow that would really please a lot of people.

David



I know that there are massive amounts of data and we all want the best quality out of the machine

David.

 

[Gunleik Groven]

I only wish for the day that color locks the data after the RED room into RGB and that the debayer was full. Then I might even agree with you on Color workflows -

As to your RR wish, I am pretty sure that it will be fullfilled.

 

[David Battistella]

I only wish for the day that color locks the data after the RED room into RGB and that the debayer was full. Then I might even agree with you on Color workflows -

As to your RR wish, I am pretty sure that it will be fullfilled.

It's a ful debayer in FCP 7.0 and Color 1.5. It's going throuhg QT but as far as I know it is full.

I'd have to test a sequence against a DPX sequence.

David

 

[Gunleik Groven]

Do...