CS6 Render Testing and Improving Performance

[Cory Petkovsek]

After experiencing slower than expected R3D renders not taking advantage of my system, I performed a variety of tests and posted my semi-subjective/objective results below. For those discussing double scaling, see tests 7 and 12.

System: Win7/64, 24GB ram, core i7 960 3.2ghz, +200mb/s disks, GTX480 w/ CUDA, Premiere 6.0.1, No Rocket, Nvidia 301.42 (Cuda 4)*;
Project: A feature film about 90 minutes comprised of 5k full frame R3Ds, 2k exr, 1080p dnxhd and other formats. GPU is enabled.
Sequence: 2048x858 (2k Scope). The R3Ds are all scaled down using the motion effect, most to 40%.

I can edit and play the timeline in realtime at 1/4. The first couple minutes of the film clips have only scaling or position effects. There is also a timecode overlay.

The only oddity so far is when I tell PPro to render it says some media is offline, but everything is online. I ignore this and continue. Offlining all media and relinking has no effect. No offline media graphic appears in the render.


Test 1: Render x264 2k scope
x264 accessed via x264vfw
Only one ImporterREDServer process ran, taking up one core; ~25% cpu.
After 20 minutes, Finish time is 50 hours and climbing


Test 2: Disable timecode; Render x264 2k
Same results


Test 3: Export x264 2k instead of Render via AME
Display is different but based on the behavior my conclusion is same results.


Test 4: Render OpenEXR 2k
6mb per frame; CPU maintains 100% with 5 ImporterREDServer processes running
After 20 minutes, remaining time: 26 hrs and climbing


Test 5: Render PNG 2k
2mb per frame; CPU 50% w/ 2 active ImporterREDServer processes;
Definitely slower than OpenEXR; aborting.


Test 6: Render Tiff 2k
7mb per frame; 100% cpu;
After 25 minutes, 25 hours and climbing


Test 7: Double Scaled EXR 720p scope
1280x536 "720p scope"; 5k R3D->2k sequence->720 render
Non R3D footage definitely faster than on 2k sequence.
R3Ds a little slower than on 2k sequence.
After 5 minutes calculating 16 hours and rising; figure it will say 30-40 hours after 20 minutes. Definitely not 10-20 times slower (i.e. 250+ hours). See #12.


Test 8: Disable GPU for project, Render EXR 2k
After 8 minutes, already at 23 hours and climbing steadily. GPU definitely provides some performance benefit (scaling and keying if nothing else). Estimate over 30 hours by 20 minutes. Compare test 4.


Test 9: 5k FF sequence -> Render EXR 2k
5k timeline scaled to 2k scope at AME render
2k/hd files are scaled from 2k to 5k back to 2k. R3Ds scaled 5k to 2k at the render instead of at the sequence level.
After 20 minutes, 14:22 remaining and slowly climbing.


Test 10: 5k Scope sequence -> Render EXR 2k
Same as above, but the sequence is 5120x2145 (1:2.39)
After 20 minutes, 13:57 remaining and slowly climbing.


*I experienced some corrupt GPU premiere compositing (pictures behind green screen footage messed up). Upgrading NVidia 301.42 -> 306.23, which also enabled CUDA 5, so far seems to have fixed the problem and a hanging problem (see Adobe Trouble Notes).


Test 11: 2k -> Render EXR 480p
872x366 "480p scope"
Actually rendered the whole thing for a submission. Completed in just under 12 hours.


Test 12: Repeat #7 double scaling w/ CUDA 5
1280x536 "720p scope"; 5k R3D->2k sequence->720 render
After 20 minutes, 26 hours remaining and climbing. After an hour 30+ hours remaining. Same results as test 7? I didn't let #7 run so all I can say is it's close, but it's possible CUDA 5 provided a minor performance benefit over 4.


Test Conclusions:
5k sequence -> Render 2k EXR: 14+ hours
2k sequence -> Render 2k EXR: 25+ hours
2k sequence -> Render 2k TIFF: 20-25+ hours
2k sequence -> Render 480p EXR (double scaling): 12 hours
2k sequence -> Render 720p EXR (double scaling): 30-40+ hours
2k sequence -> Render x264: 50+ hours
Rendering x264 off my 480p EXR/TIFF Master: 20-30 minutes.


Methods to improve performance:
- Rocket (assumed)
- Enable GPU/CUDA (upgrade to latest driver w/ CUDA 5)
- Edit at native resolution, scale down at render time only
- Export a master via EXR or Tiff sequence, then reencode to delivery format (I also apply sharpening to the master).



Adobe Trouble Notes:

1. There is a scaling problem that Adobe should fix. There isn't any difference in the amount of work between scaling 5k->2k at the sequence versus scaling 5k->2k at the render as long as it's done only once. Yet on this project there is a greater than 75% performance cost. Editing at 5k means upscaling 2k footage to 5k and scaling back to 2k is likely to cause quality loss. Either way we lose time or quality.

2. AME has (had?) hangups accessing red footage via ImporterREDServer when other Adobe apps are open. If I open Premiere or photoshop it will usually cause the AME render of R3Ds to EXR to hang. Even if I want to render a wave, AME will hang until I close premeiere before it will go. Once it is hung, I can click stop, but then it stops responding and I have to force it closed and kill the left over processes.

Update: After upgrading the nvidia driver (and cuda) this seems to have fixed the problem. I can now load AE, Photoshop, and even play the film off R3Ds in premiere, while AME is using 100% cpu rendering out the film; 16 importerredservers running. This is working just like it should, and damn impressive. The timeline/AME scaling issue is still present.



Probably this week I will upgrade to Premiere 6.0.2 and rerun some of these tests.

Feel free to post your own test results here or have discussions on performance testing and improvement.

Cory

 

[Cory Petkovsek]

Reserved for 6.0.2 results.

 

[Bob Gruen]

Your disk is slow and you did not post your CPU specs.

I'm still on 5.5. I'm using 5 2TB WD Caviar Blacks on motherboard based RAID 5. I get 440 MB/s write and 550 MB/s Read. It IS tricky to set up properly, there are various posts on the internet about how to properly align the partition and drive format to get this kind of performance. I also have the OS on SSD, and I render out to the SSD drive to avoid drive contention.

My system is a 3.2 GHz i7with a mild overclock to 3.9 GHz (Noctua NH-D14 heatsink). Clock speed makes a big difference. Especially when you see that only one ImporterRedServer is starting up (which is common for Pr). A single i7 at 4GHz can out render a dual core Zeon at 3.2 GHz because Pr cannot take advantage of all the extra cores.

I also have a red rocket, but when rendering my film I have it set up to handel the linked Ae comps and not the footage in Pr. My film is 4K 2:1, and I've been rendering out various formats and the entire film takes maybe 16 hours to complete. If it didn't have the Ae comps or RR it would probably be less than 6 hours.

The R3D files store the frames as JPEG2000 images. JPEG2000 is a very good, very adjustable image format but it is a fundamentally different type of image technology than just about everything else (wavelet). The RR card is built to process this type of math, CUDA (or other video card technologies) is not.

Bob

 

[John Tissavary]

I recently struggled with PP 6.02 when trying to render some offline material for Resolve. What I found is that using the scaling transform is where rendertimes go crazy (and also result in scaling problems when exporting XML or AAF for Resolve conforms).

If I right click > scale to timeline (or whatever it's called) the performance shoots up dramatically - and scaling in Resolve is correct. I'm not sure what the difference is, but I assume that transform scaling results in a full debayer no matter what, and 'scale to timeline' causes PP to ask the SDK for a lower resolution debayer if the timeline res is below a certain threshold. This is an assumption based on the vastly different performance between the two, and comparing that to behaviors in other software.

I believe Adobe needs to add the ability to select per-clip debayer levels like many other packages do. Without that we are left either guessing what debayer resolution PP is using, or getting stuck with a debayer level that is inappropriate for the particular task. We need to be able to choose from Full Premium down to 1/16 debayer purposely, not accidentally due to some mysterious combination of effects selections.

JT

 

[John Tissavary]

Your disk is slow and you did not post your CPU specs.

I'm still on 5.5. I'm using 5 2TB WD Caviar Blacks on motherboard based RAID 5. I get 440 MB/s write and 550 MB/s Read. It IS tricky to set up properly, there are various posts on the internet about how to properly align the partition and drive format to get this kind of performance. I also have the OS on SSD, and I render out to the SSD drive to avoid drive contention.

The R3D files store the frames as JPEG2000 images. JPEG2000 is a very good, very adjustable image format but it is a fundamentally different type of image technology than just about everything else (wavelet). The RR card is built to process this type of math, CUDA (or other video card technologies) is not.

Bob

Disk speed will only have a significant effect when rendering to EXR (or similar high-data rate formats). R3D data rate is relatively tiny, so that's not going to make any difference when rendering x264 or other compressed formats, but you're absolutely right about rendering to a different drive than you read from , always makes a difference (unless you're working from a true hardware based RAID or SAN/NAS).

I get 25-50 fps when rendering r3d with a redrocket from a single 7200rpm drive in Resolve, clearly disk speed is inconsequential.

You're spot on regarding debayering, there are only two choices - Redrocket or insane amounts of CPU power. Latest 16 core workstations will get you half premium debayer in realtime (Resolve, not PP), so actually quite good. Too bad PP isn't there yet...

JT

 

[Bob Gruen]

John,

I've seen disk speed be a significant issue in rendering performance, but it really is more of an editorial and Ae issue. I've never seen real-time rendering from a single drive though... Does Resolve fully render out the JPEG 2000 images if the destination resolution doesn't require it? When you use redline at the fractional resolutions it only has to process part of the wavelet to get the required resolution, then it skips a chunk of the file and moves on to the next image. Are you sure in your example that your computer isn't doing this?

Bob

 

[Cory Petkovsek]

@John, In these tests, I never used the 'scale to frame' option (right-clicking clip on timeline); in my experience, this doesn't work well when you shoot FF and edit scope. It adds bars. I only scaled either with the motion/scale parameter, or the scale option at render. The latter produced a two fold performance difference.

@Bob, I added my cpu. I have 4 SSDs, a 6 disk 7200rpm RAID, and 3 more 7200rpm SATA drives. In my tests, none of my disks were even close to maxed out in transfer rate nor queue length. I will look at partition alignment to see if I can get a greater disk speed.

You mentioned CUDA does not do what a Rocket does. Of course not. Though there is a tested performance benefit when using it; premiere uses it for scaling, keying and a variety of other tasks.

Whether my disks are fast or slow is not important, nor is the actual time of each test. What is important is the relative time differences between my tests, and that all my tests are done with the same project, and same disks.

Cory

 

[Cory Petkovsek]

I'm using 5 2TB WD Caviar Blacks on motherboard based RAID 5. I get 440 MB/s write and 550 MB/s Read. It IS tricky to set up properly, there are various posts on the internet about how to properly align the partition and drive format to get this kind of performance.

Bob, That is an impressive rate. Is this sustained? What is the raid chip on your motherboard? Could you test with HD Tune and post a screen shot of all the results please?
http://www.hdtune.com/

Cory

 

[John Tissavary]

In most grading software (like Resolve, Scratch, Speedgrade, etc...) you can choose to debayer at a partial resolution setting (1/2, 1/4 etc...) but that can be modified on a clip by clip or project basis at any time: debayer resolution is not linked to project or timeline resolution.

In Resolve, as in Scratch, when set to Full Premium debayer (highest quality, entire r3d frame data) with a single redrocket I'll get 25-30 fps from 4k, 15-20fps from 5k Epic (2x Redrocket needed for 5k Epic realtime). These are on single drives (7200rpm, bare drives in sledless eSATA enclosures).

The Redrocket also does the scaling in hardware, so it will supply the fully debayered, scaled RGB data to the software client through the SDK. If the software client (Resolve, PP, etc...) is asking for 1920x1080, the Redrocket first debayers all the frame sensor data, then supplies a scaled RGB frame in that resolution. There are different implementations of this - for instance Scratch has a project resolution setting in the SDK parameters that will concatenate

Seems to me like PP / AE are not allowing access to all the Red SDK settings, and in some cases where it's not working right are communicating with the SDK incorrectly. Every single application out there uses the exact same SDK from Red. It is a violation of Red licensing to do otherwise. So potentially, all applications can offer the same settings & debayer performance (not including what happens once the RGB data is supplied to the application), but that is apparently not the case here.


JT

 

[michael zaletel]

Thanks for all of this!

Michael Zaletel
Founder, i4software

 

[Viktor Sloth]

Thanks for this, Cory!
May I ask how much your GPU are working?

I'm spitting a music video out now with loads of effects and the CPU are at 100 % constantly, but the GPU only does 'something' occasionally, and when it does it's like 25 % for 1-2 minutes.

I have somewhat the same system as you:
i7 3770K overclocked to 4,3ghz
32 gb ram
SSD drives
GeForce GTX660TI - Cuda enabled by script.

 

[Stefan Scherperel]

Cory, you should be able to set up a custom timeline for your files to scale to 2K and check scale to frame size. In my experience using scale to frame size and unchecking render maximum render quality provide the best looking and fastest rendering results. Please see my tests here that I did a few months back.

http://www.reduser.net/forum/showth...manual-scale-vs-auto-scale-tests-LARGE-IMAGES