6 ROCKETS and REDCINE-X PRO v9

[Michael Cioni]

If you're like me, it's no secret that the trend of higher shooting ratios is on the rise. With more cameras, more coverage and more footage to process, the need for additional speed may be welcomed by independents and facilities alike. Recently, my friends Torrey Loomis from Silverado Systems and Eric Fiegehen from Cubix helped us out by allowing us to test the latest Cubix XPander (http://www.cubix.com/products/gpu-xp...nt/rackmount-2) with REDCine-X Pro v9. We then tested the Rocket acceleration performance with 1, 2, 3, 4 and 6 Rockets in a MacPro (read and write disks were separate volumes). And thanks to Rob and Matt at RED for their advisement and hard work in optimizing RCX-Pro v9 performance. After some extensive testing, the Cubix XPander is a great tool for people who are looking to increase their REDCine-X Pro transcode performance to be much faster than real-time.


If you routinely render ProRes, DNx or DPX files, the below chart will show you what an investment in such technology will buy you. You'll see that the sweet-spot for most of these processes is about 3 Rockets. After you go up past 3 Rockets, performance decreases significantly due to bus limitations, disk I/O limitations and encode limitations (on compressed codecs such as ProRes). But you'll also notice that the results across different codecs is encouraging since the encode times are very similar.


EPIC 5120x2700 4:1
internal raid 0 (560MB/s) to sas (580MB/s)

69.5 fps 1080 prores PROXY 4x rockets
59.4 fps 1080 prores PROXY 3x rockets
43.7 fps 1080 prores PROXY 2x rockets
23.2 fps 1080 prores PROXY 1x rockets

63.1 fps 1080 prores LT 4x rockets
58.3 fps 1080 prores LT 3x rockets
43.8 fps 1080 prores LT 2x rockets
23.1 fps 1080 prores LT 1x rockets

65.7 fps dnx 36 4x rockets
59.9 fps dnx 36 3x rockets
43.5 fps dnx 36 2x rockets
23.1 fps dnx 36 1x rockets

64.6 fps dnx 115 4x rockets
60.1 fps dnx 115 3x rockets
43.6 fps dnx 115 2x rockets
23.3 fps dnx 115 1x rockets

44.5 FPS 1080 10 BIT DPX 4x rockets
44.1 FPS 1080 10 BIT DPX 3x rockets
42.7 FPS 1080 10 BIT DPX 2x rockets
23.3 FPS 1080 10 BIT DPX 1x rockets

[Steve Johnson]

Wow. Amazing tests here Michael.

So I take it that the expander connects to a MacPro x16 slot and from that, 4 Rockets gives these performances. I guess for speed, this is going to cut through footage like a knife through butter. Did you do any tests in how this would cut through footage for 3D work?

:))

[Michael Cioni]

3D would be the same results in terms of fps (within a couple of percents). You can mux 2 R3Ds at approximately the same speed so long as your network is able to read 2 streams of R3Ds. That means 2 rockets will render 3D in just short of real-time.

m

[Bruce Allen]

Awesome fun!

Interesting that DPX is slowest - due to drive write speed limitations obviously because DPX files since they're not hard to encode? Those DPXes are just too big!

If you routinely render ProRes, DNx or DPX files

Two questions:

1. How about OpenEXR, B44? And B44 in 4:2:2 mode? Come on Michael - you know I was thinking of you and your 54TB of disk space for TGWTDT when I asked Brendan to write that ProEXR importer / exporter with timecode for AE and Premiere :)

2. Does that sucker work in Resolve?

Bruce Allen
www.boacinema.com

[Subhadip Sen]

Interesting! The Mac Pro + Cubix solution is extremely PCIe bandwidth starved, however, which is the bottleneck past 3 Rockets even though Rocket is not as hungry as modern GPUs. (Of course, for DPX it is disk I/O) Will be interesting to see 6 Rockets going on a Sandy Bridge-E/EP system.

[Stephen Lovett]

Thanks very much for this Michael.

It is something that I've been interested in but frankly its not many of us have the resources to test the permutations to this extent.

I've been holding out for a thunderbolt based expansion chassis to allow both cart based and portable (MacBookPro) scenarios.

But... that limits total throughput to Thunderbolt throughput.

Perhaps I need to rethink that.

Is it your thinking that its PICe itself that is the bottleneck here (or one PCIe 2.0 x 16 lane in the MacPro), disk I/0, resource marshaling, or is there enough information to pin that down?

This is super cool, maybe even cooler than the new outdoor kitchen :-)

Steve

[Subhadip Sen]

Thanks very much for this Michael.

It is something that I've been interested in but frankly its not many of us have the resources to test the permutations to this extent.

I've been holding out for a thunderbolt based expansion chassis to allow both cart based and portable (MacBookPro) scenarios.

But... that limits total throughput to Thunderbolt throughput.

Perhaps I need to rethink that.

Is it your thinking that its PICe itself that is the bottleneck here (or one PCIe 2.0 x 16 lane in the MacPro), disk I/0, resource marshaling, or is there enough information to pin that down?

This is super cool, maybe even cooler than the new outdoor kitchen :-)

Steve

Since this test is on a Mac Pro with a Cubix Xpander, PCIe bottlenecks are natural and expected. Rob has previously confirmed the PCIe bandwidth will be the bottleneck: http://reduser.net/forum/showthread....l=1#post906633. With a more suitable system we could see 6 Rockets scale.

Thunderbolt offers 1/4th of a PCIe 2.0 x16 port, so it would be suitable for a single Rocket at best.

[Mark L. Pederson]

With a more suitable system we could see 6 Rockets scale.

And such a system exists in the skunk labs over at JMR right now :)

[M Most]

I understand the fun of a technlogy demo, but for the price of 6 Rocket cards, I coukd buld at least 12 reasonably hefty PC's, which would not only allow similar (or better) transcoding performance for dailies purposes, but would do a whole lot of other things, and for files ofher than R3D. I understand the demo purely for curiosity's sake, but Red Rocket cards cost almost $5000 each and only do one task with one type of camera file. This approach seems pretty cost intensive for a single purpose solution. To me, a render farm is much more sensible, practical, and versatile.

[KETCH ROSSi]

He he, love you for it Michael... ;)

I understand the fun of a technlogy demo, but for the price of 6 Rocket cards, I coukd buld at least 12 reasonably hefty PC's, which would not only allow similar (or better) transcoding performance for dailies purposes, but would do a whole lot of other things, and for files ofher than R3D. I understand the demo purely for curiosity's sake, but Red Rocket cards cost almost $5000 each and only do one task with one type of camera file. This approach seems pretty cost intensive for a single purpose solution. To me, a render farm is much more sensible, practical, and versatile.

I do see your point, a valid one up to a certain point, more machines also a lot more problems to come, but also if one goes down here you have more, but problem with your solution is that by by portability any were... ;)

Oh and did I mention I absolutely despite PC's... Sorry.