Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

New Mac Pro w/ Red Rocket-X in Thunderbolt 2 vs Old School Mac Pro

M

Michael Gregory

Well-known member
Joined
Jan 30, 2014
Messages
70
Reaction score
0
Points
0
I have been waiting on my nMP (New Mac Pro 2014 model) for almost 4 months. I had ordered a TB2 (thunderbolt 2) Raid and Sonnet PCIe Expansion Chassis. I looked specifically for a chassis that had 16 lanes. However, there are 16 mechanical connections but only 8 electrical and only 4x effective after going through Thunderbolt. Even though TB2 is twice as fast as TB1, there are only 4 physical lanes of traffic going over Thunderbolt. While TB2 is great for moving storage data it cannot handle the bandwidth of real time graphics computations where gigabytes of information are flowing back and forth every second. I will save the technical discussion for another time and talk more about why I choose to cancel my nMP (12 core, 64 Gb, Dual D700s) in favor of a souped up 2012 model Mac Pro (Old MP).

The nMP has a faster internal bus and the graphics cards are running at PCIe 3 speed but the 12 core only runs at 2.7 Ghz. It only connects to the RRX via a 4x PCIe connection. Even when you install the RRX in the 16 "mechanical" slot in the Sonnet box, RCX gives you a warning that it is running in a 4x slot and that the RRX requires a 16x slot to perform optimally. Seems crazy to go to the expense of having 6k footage and a RRX and not get full performance. However, the nMP also has a 1Tb solid state drive that can deliver 1,000 Mb/s and 64 Gb of RAM.

I found someone (Edit Builder, you can find them on ebay) that would sell me a Mac Pro 5,1 (this is the most recent build of the old Mac Pro) with two 6 core 3.46Ghz (12 cores), and 128 Gb of Ram. I then mounted 8 6gb/s eSATA drives connected to Acera 8 channel raid card. This gave me twice the main ram memory and 4x the Solid State storage. Additionally, my RAID array can sustain transfers that are almost twice as fast as the nMP using its SSD. The maximum SSD storage on the nMP is 1 Tb, which after the OS, the usual programs and some data barely gives you enough room for an hour of 6k footage. Since I use FCPX heavily, I needed video cards that could push the OpenCL architecture of FCPX at least as well as the FirePro 700s in the nMP. However, the last AMD video card that Apple supported was the AMD 5870. This card is more than 5 years old and performs like it.... basically useless at a professional editing pace. I found a company (MacVid cards, also on ebay) that sells flashed 7970s. I installed one of these and FCPX performance really improved.... the spinning color wheel all but went away. However, I also knew from reviews done on Barefeats.com that two cards is better than one, especially in FCPX 10.1. Then I heard about the recently released R9 280x that can be booted under Mac OSX. But, I wanted two of them AND my RRX. So I found a company Cubix that sells a real 16 lane PCI expansion chassis. These use a host controller that is connected to an external box about the size of 3 nMPs and connected by 4 cables about the size of jumper cables, no flimsy TB cable here. So if two GPUs is good.... then 4 must be better....

The build:
MAIN PROCESSOR:
12 Core 2012 Mac Pro 5,1 with 3.46 Ghz xeons (2 - 6 Core Chips) this was never available from Apple.
128 Gb of 1333Mhz Ram
Samsung 1 Tb SSD - Boot drive for OS and Programs
8 Swift Data Synchronized 6 Gb/s eSATA Solid State Drives (480 Gb each) ((this is where the working footage lives))
Areca ARC-1882i Raid Card (slot 4)
Nvidia K5000 Quadro Mac (slot 1)
Cubix Destop Elite GPU Expander (slot 2 - the host adapter MUST go in this slot, MacPro only has two 16 lane slots)
Sonnet 6Gb/s eSATA controller (to connect Red Mag Reader via eSATA) (slot 3)

CUBIX: Expansion Chassis
RRX in Slot 1
Three Radeon R9 280x in Slot 2, Slot 3, and Slot 4

This configuration allows the Nvidia K5000 Card to handle all the GUI and Display functions PLUS it is the only video card I could find that would actually output to a 4k monitor. The Radeon cards work fine on 27" Cinema Displays but will NOT drive the new Sharp 32" 4k monitor in Mac OSX 10.9.2

RESULTS:

So far on Open CL benchmarks it is 2.5 times faster than the best benchmarks for the nMP. It will play back 6kFF footage at 29.97 in real time in RCX and FCPX (I haven't shot anything any faster yet). But I can also use the "JKL" keys to speed it up to over 45 fps.

So what kind of idiot goes to all this trouble, I use my Dragon for live action footage of sports. Sometimes I have 4 hours worth of playing time that occurs on a weekend and needs to have graphics (scores, stats, etc) rendered to bluray and burned to 60 different disc... Oh.... by that Monday. I can't spend days or even hours rendering footage, for me it needs to be real time or faster. I can tell you that this set up (God Bless the RRX) makes that possible. I rendered out ProRes 4444 4k HD film that was 61 minutes in under 45 minutes in FCPX, keep in mind there are 6 tracts of continuous motion graphics and fully graded color. I tried this same maneuver with my 2013 Retina MacBook Pro via a TB2 connected Sonnet TB2 enclosure with a RRX inside it. I don't know how long it would have taken but after 2 hours and still being around 10% complete I pulled the plug.

I don't know if this helps but I will say that the RRX is amazing and if what I've read about it only using 1 of its 3 cores is true it seems silly to put it into a box that will only utilize a fraction of its potential.

Below are some links you all might find helpful.....

Edit Builder MacPro Test: http://www.barefeats.com/mp12c346.html

nMP vs Edit Builder 3.46 Ghz: http://www.barefeats.com/tube01.html

Raid Card: http://www.barefeats.com/hard174.html
((keep in mind the newer one is faster and I am using 8 drives which doubles the theoretical bandwidth))

nMP vs Old MacPro (upgraded video card): http://www.barefeats.com/tube11.html

Just as an example the nMP can achieve 3528 Ksamples/sec on LuxMark (OpenCL Benchmarking program), while my machine hit 8768 on the same benchmark (more that 2.5 times faster).

This is just one way to skin the proverbial dead cat, but I like the idea that I can change my storage and my cards when and if my needs change. The nMP in my mind is really just a very expensive MacMini with more processing power. If you order one you better like it because you can't change it once its built.

If anyone has specific questions please post and I will try to respond intelligently...

MWG
 
Thanks for sharing
 
Thanks a lot for sharing. Very interesting!

I love Apple afte being a PC guy most of live, but I do find the hardware frustratingly limiting with the new models.
How good does the computer actually work? I'm used to when building my own configurations there will often be issues, drivers not supported, or specific features simply not working in some programs.

Cheers!
/Andreas
 
Is there TB2 expansion chassis that can make full use of RRX with the nMP? Also have a nMP on order but the OP gets me thinking

Like this perhaps:
http://www.magma.com/roben-3tm
 
Last edited:
A few questions

How did you setup the Raid again. Did you have an Acera 8 raid case or is it just a controller card?

Is it possible to connect thunderbolt in any way to such an Apple? I have 4 Raid thunderbold drives that would be nice to be able to keep using if possible.

Are there any gpu expansion boxes that are less expensive. The Elite box cost $3300 which is quite substantial.

Very interesting solution, will consider something for our next major purchase.
/Andreas
 
Zeb B said:
Is there TB2 expansion chassis that can make full use of RRX with the nMP? Also have a nMP on order but the OP gets me thinking

Like this perhaps:
http://www.magma.com/roben-3tm
Click to expand...

As I understand TB2 can never use the RRX to anywhere near its potential I'm afraid. The speed in PCI 16 generation 2 is about 64Gb/s while Thunderbolt 2 is 20Gb/s (theoretically, but probably less in practice). However as I understand on the Mac Pro its actually less than 20Gb/s because it only uses 4 lanes for the information. I don't think there is so much point in having a Rocket card with the new Mac Pro, i'm sure it will be faster but not so sure you want to spend so much money for it.

Simply put the RRX is not very useful on Mac Pro sadly, however since you have dual GPU you will get increasing speed when more and more programs use the GPU accelerated debayer settings (like Davinci does for example). The OP however uses 4 fast graphics cards AND a Red Rocket X.

Of course the hacked Super Mac Pro is about 3 times as expensive (when you add the Rocket). The setup is also MUCH bigger and heavier which could make a difference if you travel.

/Andreas
 
As I understand it, expansion chassis mfgs. are exploring units with 2 or more Thunderbolt lanes inputing to the chassis so that certain cards, such as the RedRocket X can be utilized to their full potential.
I saw an article somewhere how the TB connectors on the new Mac Pro are paired. Lane 1 = top left & middle left connectors. Lane 2 = top right & middle right connectors. Lane 3 = bottom left & right connectors.
Running 2 or more TB lanes into a multi-lane expansion chassis should speed things up quite a bit.
Chassis manufacturers?
 
Randall Weatherington said:
As I understand it, expansion chassis mfgs. are exploring units with 2 or more Thunderbolt lanes inputing to the chassis so that certain cards, such as the RedRocket X can be utilized to their full potential.
I saw an article somewhere how the TB connectors on the new Mac Pro are paired. Lane 1 = top left & middle left connectors. Lane 2 = top right & middle right connectors. Lane 3 = bottom left & right connectors.
Running 2 or more TB lanes into a multi-lane expansion chassis should speed things up quite a bit.
Chassis manufacturers?
Click to expand...

That would be great if it would be possible!
All this would have been saved if Apple had an PCI connection that you could use on the Mac Pro. Then we could just put in expansion chassis with all our toys. I'm still gravitating towards the new Mac Pro since its cheaper and so much smaller and I'm not rendering THAT much (yet).
/Andreas
 
A chassis with 2 Thunderbolt lanes would be great.

Is anyone trying that out in reality? Details?
 
I've suggested the dual TB2 I/O from expansion chassis to NeoMP on other threads. In the current iteration of the NeoMP even a dual TB2 solution will still only have 8 combined PCIe 2.0 lanes, much better than 4 lanes but still underwhelming for harnessing the power of a card like the RR-X, Titan, K20, etc - not to mention several of such cards running at the same time.

Assuming its not silly expensive, I'd definitely want dual TB2 I/O to an expansion chassis - and not just to break the 4 lane bottleneck - but also in hopes that the next gen of the NeoMP will have new CPU/socket architecture that provides PCIe 4.0 lanes to allocate to TB I/O. Will be interesting to see if bonded/multiplexed TB2 bundling comes along - that 20gb/s cap is superfluous for 99% of applications, but for RR-X and some other bomber cards even 20gb/s can be a limiting factor you don't want.

FWIW, I had hoped that Apple would offer a "base" for the NeoMP that looked like a big hockey puck, had a multi pin connection to the bottom of the cylinder, and iPASS ports for expansion. No, I'm not holding my breath.

Cheers - #19
 
Late to this thread but thanks for posting!
Great insight into how much performance can be squeezed out of the previous generation.
Will look into Edit Builder for upgrades also.
 
Ive setup 6 nMPs in the last month for R3D workflows.

Basically puts the ROCKET away due to the Thunderbolt 1 option (all we have).

Did some tests with Transcodes to DNxHD 175 from 4K HD and were off by 5% in speeds compared to using the ROCKET in a Magma ExpressBox 3T.

Cant speak for ROCKET X or Thunderbolt 2 though.

The new Mac Pro's rock in so many ways.
 
A 12c/D700 Mac Pro will do a half-res decode of 6K footage at 30+ fps on the GPU, which works just fine for grading workflows where you're monitoring 1080p/2K live and only moving up to a full-res decode on output to generate a final deliverable (i.e. most grading workflows in use today), and works just fine for kicking out 1080p editorial dailies at a reasonable pace. Even full-res 6K comes pretty damn close to real-time (19-21 fps), so those final renders aren't all that bad. This makes a Rocket X a pretty questionable investment for most users, I think. Especially with the pace at which GPUs improve. We've gotten ~5 solid years of usefulness out of our original Rocket. Thinking about what GPU performance will be like in another two years makes me really wonder whether a Rocket X will be useful for nearly that long.
 
OK... I am not trying to start a contest. BUT at 6k ful-res on a Mac (I have not tested this on Windows). A Rocket X is a game changer.... I run 4 of the fastest graphics card available and the RocketX changed my world. So much so when my first one (an early version) failed it was hard to use. Also I have a larger budget than most so I bought a top of the line Mac Pro...If you want it... it is for sale.

I agree that the future of profession post is Adobe of something like it. Not Mac. I love the OS but Steve is dead and so is the product. If you are buying new equipment buy a windows machine. Adobe runs the same, it is harder to use but you get used to it. I spent a fortune on trying to stay with Mac but they would rather make phones than computers and certainly don't care about hardware for the professional.

Peace,

MWG
 
Can't believe that was you only option. I have using a mac version of the nVidia GTX 680 and there are some faster options now. You wanted AMD specific GPU for OpenCL?


Michael Gregory said:
However, the last AMD video card that Apple supported was the AMD 5870. This card is more than 5 years old and performs like it.... basically useless at a professional editing pace.
Click to expand...
 
I build grade suites for a living. We are using souped up OLD Mac Pros with CUBIX expanders and 4 very fast GPUs on FC SANs. Sometimes with RRX cards (FCPX is still RRX/software debayer), sometimes not. We are not using NewMacPros. The performance isn't there.
 
Update on Old School Mac Pro

Update on Old School Mac Pro

Thank you for all your kind comments. I am still trying to improve on this and have some real world results to share with you all.

Current Configuration:
Same basic hardware but I swapped out the Nvidia Quadro 5000k for a 4th Radeon R9 280x now in Slot 1 of the MacPro. Also I changed to the 12 channel Areca ARC-1882xi Raid Card with 4Gb of on board RAM. I also swapped out the 6G eSATA card (since i know have 12 channels of 6gb eSATA on the raid card) with a Compress HD card from Matrox ( http://www.matrox.com/video/en/products/compresshd/ ). This has historically allowed a virtual 1:1 encode rate of h.264 files, by this I mean 1 hour of video takes 1 hour to encode.

Results and Workflows:
This is where i gets interesting. Historically I used an older 8 core MacPro with the CompressHD card and would transfer the files but even on dual Gigabit (using LAG protocol, effectively giving each machine two Gigabit ethernet ports) transferring 480Gb of data would take about 38 minutes. So I moved it to my primary rendering box (the subject of this thread). My workflow is as follows transfer 400Gb off of the new 512Gb mini-mag via 6Gb eSATA connection takes about 23 minutes but I have two and have seen sustained transfer rates of 500Mbits (not bytes) a second for both. I think the cards probably max out about 250-300Mbits/sec. I am transferring to 3Tb SSD Raid array with 6 drives (this is where the Raw files go). Using Blackmagic disk test utility, I have seen speeds in excess 3000Mbits/sec. So if you are going to read two MiniMags at a time make sure the drives you are transferring the data to are FAST as the new cards are 30% faster than the old Turbo 512Gb cards. Now I've got the Raw files on my internal SSD Raid. I shoot mostly sports at 5k and 60fps at 12:1 compression. Red CineX can render this at full debayer on my system at almost 60fps... it doesn't seem happy about as I have to use the "JKL" keys to get that speed and I may be dropping frames (I am not sure how to tell inside Red CineX). I do some minor corrections to the color/exposure, I personally am VERY happy with RedGamma4 and DragonColor, then save the RMDs. Then I open them in FCPX (yes I am back to this AGAIN... more on that at the end). I then edit, add graphics, titles, add surround sound, etc.

Currrent WorkFlow:
I export this to a 4444XQ file (1 hour of Video = about a 430Gb file) at 1080p-59.94fps. I then open Compressor and using the Matrox Presets render this to a h.264 and an ACC audio file (5.1 surround). This is encoded at a constant 40Mbs which is the limit of the BluRay spec. Most theatrical productions are only encoded at 30Mbs, some get up to 37-38Mbs but there are only a few 40Mbs titles out there. This produces about 18Gb of video file size (the h.264 file) per hour of video. IMPORTANT: If you do use these settings the h.264 encode must be set to level 4.2 which will NOT play in older BluRay players. I use OPPO players and they work great, also works on a PS3-4. I then use Toast Titanium with "re-encode" set to "never". I have tried everything to get these to burn at faster than 1x but because of the amount of data and the density of that data I am seeing speeds on 12x drive of about 2/3x which means it takes about 90 minutes to burn 1 hour of video. Since single layer BD-R (recordable Blu-Rays) only hold 25Gb you can get about 90 minutes at this setting.

Discussion:
The problem for me is the multiple steps and transcoding. The prevailing wisdom is "just switch to premier". I have TWICE. Here's what I ran into the last time. While Premier doesn't require you to transcode before going to Adobe Media Encoder, neither does the FCPS-Compressor workflow. BUT... When I tak my files straight to compressor (make sure you shut down FCPX before you start encoding if you have a Rocket because for Compressor to use the Rocket, FCPX must release it first). This seems like a great workflow but my encode rate is about 9-12fps which on a 60fps project that is an hour long... it takes to long. THIS PART IS A GUESS: I believe that in the above "straight to compressor" workflow that neither the RocketX nor the CompressHD cards are maxed out. This means they are waiting on each other. The result is about twice to 3 TIMES the amount of time to get to a file ready to burn on a BluRay. I tried this in Premier and the results are similar. However, Premier doesn't support 5.1 surround in quicktime without a plug-in AND the CompressHD program while it works with Adobe Media Encoder it does not support BluRay encoding... must be some copyright issue.

Want List: One step process between RedRaw and files that can be encoded to Blu-Ray. Don't RRencode... because I tried that....bought a RedRay and everything (which they don't support anymore) by the time you export to TIFF, fire up Red CineX v20 then encode it.... it takes HOURS, I am talking like 12 hours to do an hour of video at 4k. Not worth it to me... Probably for feature films but in my world, I shoot predominately live events that need to be ready the next day.

I would LOVE your comments and questions. Again thanks to all that have shown an interest in this thread.

Michael
 
Thanks for the notes Michael, you have tried and tested some things I've been considering. I assume your "want list" item about a one step R3D to H.264 workflow is actually a desire to do so at faster than the 9-12fps you're seeing with the RR-X and CompressHD hardware working simultaneously. You might be correct about the cards having to wait for each other, IIRC even a one pass H.264 encoding does some motion estimation analysis and "I" frame management so it may need "chunks" of a certain size in a buffer (or otherwise accessible) to perform that part of the process. As a guess, I'd suspect that when you offer the Matrox card a file to encode it's doing quite a bit of "look ahead" analysis in the background so that it can feed the last stage of the encoder a real time data stream. With the RR-X sending out analyzable RGB video on the fly it may be severely limiting the use of "look ahead" strategies. There might be a way to create a custom topology that gives the RR-X a "head start" of a couple of minutes to feed RGB video into a buffer that the Matrox card can access and encode in real time. It might require a pretty decent coder to set up, but if it worked you might be able to trade a 5 minute lag in the beginning for 60fps speed during the rest of the encode. Just a theory obviously…

PM me if you'd like to dialogue.

Cheers - #19
 
Blair, that makes a great deal of sense. However, that is out of my comfort zone. When I analyze the images from the CompressHD card they are superior to any software based h.264 I have tried. I am very excited about h.265, VP9 and other future codecs that will allow us to publish 4k content. I am wondering with all this horsepower if I can go back and tweek the software setup for the h.264 in compressor and get results closer to the CompressHD.... I guess I can try but if anyone has done it already I would appreciate your thoughts.

MWG
 
You must log in or register to reply here.
Back
Top