Welcome to our community

Be a part of something great, join today!

Register Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

Expanding PCIe Slots - Good or Bad Idea

C

craigjkharris

Well-known member
Joined
Sep 5, 2008
Messages
165
Reaction score
0
Points
0
Location
Vancouver, BC, Canada
With the limited slots available on a Mac Pro, the option to expand the number of PCIe slots with something like the Cubix Xpander would be helpful.

My question...
Can a Mac Pro tower handle any more than 4 PCIe cards?
By adding something like a Cubix Xpander, will you get the performance needed from each PCIe card if you have more than 4 installed and transferring data?

Thanks
Craig
 
Thanks Dusty.
My concern is wether or not I'll be able to get full throughput if I expand the number of PCIe cards I run through the machine.

Example:
- Card Slot 1 (nVidia beasts & eSata via Cubix)
- Card Slot 2 (Black Magic Decklink X-Treme)
- Card Slot 3 (ATTO Raid Card - 16TB w/ Mini SAS)
- Card Slot 4 (Red Rocket)

If I were doing a color session using everything above, will I still get full throughput and real time performance? Can the Mac Pro handle it?
 
The Cyclone has a x16 v2 PCIe expansion, so it has lots of bandthidth, but there most be some performance loss using it. I haven't seen much difference, so the loss seems minimal to me so far.

For Resolve with the Cyclone, I would consider this config:

Mac Pro
Slot 1: GPU
Slot 2: Cyclone
Slot 3: Blackmagic (its only a 4x card)
Slot 4:

Cyclone
Slot 1: Atto (x8 v1 card, only 3G cards work right now)
Slot 2: Red Rocket (x8 v1 card)
Slot 3: Secondary Video card for display
Slot 4:
Slot 5:

And that leaves room for three more cards. Plus the enclosure for the Cyclone can be set up to hold several drives to connect internally to your Atto raid card.

All your cards are getting their full bandwidth.

I was getting just around 900MB/s read and write to my Atto R60F card in the Cyclone, but it kept locking up, so I am replacing it with an R30F. I am getting just under 500MB/s read and write with my Rocketraid 2322 I currently have in the Cyclone and thats about as fast as that card can go. Red Rocket seemed slightly slower in Cyclone than in tower, but I need to do more testing when I have time.

Someone else was concerned about the cost of a PCIe expander vs buying a seconf system. If you consider I have my raid in the case and my LTO4 in there, I could easily spend $1000 on a raid enclosure and $250 on an LTO enclosure. That is half the total cost of the Cyclone and I need those things anyway.

I am very happy with my PCIe expander. Hope that info is helpful.



Dusty
 
Like Dusty, I too am very happy with my Cyclone Microsystems 600-2707. I'm only in my second week, still on my honeymoon, but I've had this thing transcoding REDRocket dailies almost 24x7 since I got it, and so far I've experienced no significant issues (a previous thread on this forum revealed some issues with a certain workflow, but they were not caused by the Cyclone).

In any case, for those interested, here is my config:

MacPro Tower:
Slot 1 - GPU
Slot 2 - Cyclone HBA
Slot 3 - Kona LHi
Slot 4 - Sonnet Temp SATA E4P

Cyclone 2707:
Slot 1: ATTO R380 SAS Controller
Slot 2: ATTO UL5D SCSI-3 Controller
Slot 3: REDRocket
Slot 4:
Slot 5:

Best,

- Jordan
 
Anson Fogel said:
How loud is the chassis, and how far do you think you could install it from the tower?
Click to expand...

The chassis is LOUD! We're talking louder than my MacPro and two RAID towers combined. We're talking small air conditioner loud. On the plus side, it does stay very cool. I'm told however that replacing the stock fans can make it become almost silent.

- Jordan
 
The default fans are stupid loud.

My cyclone is very quiet even with 10 drives. My Mac Pro is much louder. I replaced all the fans. Actually, i replaced the entire case and everything.

I think 6' was the farthest it can be from computer.


Dusty
 
Anson Fogel said:
How loud is the chassis, and how far do you think you could install it from the tower?
Click to expand...

Oops, I forgot to answer your 'cable length' question. The default cable supplied by Cyclone is 1 meter. For a bit more, you can 'upgrade' to a 3 meter cable. Cyclone talked me out of this stating that the longer cable, the more latency. Cyclone do not officially support a cable length of more than 3 meters. Personally, I went with the 1m cable to be on the safe side, and theoretically, maximize performance.

Dusty: I saw the pictures of your 'modified' Cyclone - very impressive! I think I will have to at least replace my fans if I plan on getting this thing anywhere near a set...

- Jordan
 
PCI-E expansion chassis are probably the future for DIT setups. Apple seam reluctant releasing a Macpro with more than 4 slots. But even then the number of cards per system are still limited and I'm not sure if more slots means more lanes and how they would be shared.

Depends what kind of system you need to develop.
If you want to have a mobile grading station that is laptop based and easy to transport from set and connect to editorial this could very well be the better solution:

http://www.istoragepro.com/prod.php?id=id8exp3

You can have a Redrocket and a Rocket raid as well as several HDDs installed connected via the Express34 slot to your MBP. At rap disconnect the unit and re connect to editorial via the PCI-E HBA copy over all your data from the day and set it on ~real-time dailies compress.

But if you don't need the mobility of a smaller system then perhaps the 2707 as Dusty has posted is very good and has more slots.
 
Nir Shelter said:
Apple seam reluctant releasing a Macpro with more than 4 slots.
Click to expand...

Kind of strange since it feels like the video heads are the only ones still buying Mac Pros and we all need more than 4!

Everyone else uses an iMac or laptop.
 
Apple is not RED. not even close.

Apple is not RED. not even close.

jimhare said:
Kind of strange since it feels like the video heads are the only ones still buying Mac Pros and we all need more than 4!

Everyone else uses an iMac or laptop.
Click to expand...

Apple is not RED. so they give us whatever they Want not what we need or want even though we pay premium for it. the fact is mac pro users always have to sacrifice something because there are no more Open slots. for instance you could not have A Video card, Red Rocket Card, A Raid Card, a NIC, and a I/O card.
I have never understood the racional behind this limited number of slots.
is it that apple is stingy? or is it that they want to limit the number of different cards installed therefore eliminating potencial compatibility issues?.
 
Whoa, Nir introduces something I was not aware of - http://www.istoragepro.com/prod.php?id=id8exp3

Very interesting - and far more attractive to me than just the cyclone - we get mobile rocket capability as well as PCIe expansion at home. Anyone else used one? Or am I thread jumping?
 
The iStorage Pro chassis is essentially the same piece of hardware offered by Maxx Digital as the "Mobile Rocket RAID." I love Maxx Digital and all of the extremely helpful and dedicated people who work there. I also own the original "Mobile Rocket" and am reasonably happy with it... That being said, I don't think this product is equal in most respects to the Cyclone 2707.

I highly recommend contacting Maxx Digital (they provide outstanding service) and talking to them about your specific needs. They have always been extremely honest and forthcoming with me about their products' strengths, compatibility, and any known limitations.

Ultimately, the reason I went with the Cyclone instead of the iStorage Pro or the Maxx Digital enclosure is because they told me, money back, it would work with my existing PCI cards and I would be happy with the speed / performance. So far (1-week in), I am.

As I write this, my MacPro is currently transcoding REDRocket dailies while I also edit 1920x1080 APR422HQ in Final Cut reading from a SCSI-3 RAID-5 while I monitor out of my Kona LHi at the same time. Both the REDRocket and the RAID controller are in the Cyclone. The AJA is in the Mac. No hiccups so far.

Best,

- Jordan
 
Jordan,

Thanks. The Cyclone is not however portable/ExpressCard capable for interface as I understand it. And the LOUD thing...I am currently torn between cyclone/mobilerocket and just getting a 2nd MP Tower, but that probably is another thread altogether.
 
Anson Fogel said:
The Cyclone is not however portable/ExpressCard capable for interface as I understand it. And the LOUD thing...I am currently torn between cyclone/mobilerocket and just getting a 2nd MP Tower, but that probably is another thread altogether.
Click to expand...

The loudness is supposedly surmountable with new fans (inexpensive fix). The portability is definitely an issue, if that is important to you. However, I have to question the effectiveness of using cards designed for 4x and 8x slots in a 1x (i.e. MacBook Pro) setup. I can't imagine more than one card would even work at a time, but I could be wrong.

I also thought long and hard about just getting a 2nd MacPro tower, but when it came down to it, that was ultimately a much more expensive solution; especially given I would have to duplicate many of my existing PCI cards to make both machines effective for me. The expansion chassis was the most cost effective solution.

Best,

- Jordan
 
I have heard good things about iStorage Pro. I was thinking about getting one and still might if I can figure out how to configure it. I want Rocket, Decklink, raid, and esata, but that is one too many. Maybe Rocket, QIO, raid, and MXO2 connected through QIO. But like Jordan says that is a lot of stuff connected to a single 1x slot in a laptop.

The Cyclone 600-2707 can be made very quite. Mine is quieter than my Mac pro and I have extra fans in the Cyclone. My Cyclone is also pretty mobile. In a cart that I roll all over the place on set. Goes on location with me all over. Not as mobile as a laptop, but if I am using my laptop, all I am doing is downloading and verifying footage. I'll worry about rendering and heavy lifting once I get the footage back to my cart.

Price wise. The Cyclone is $2500. I spent a couple hundred extra dollars and got a completely new enclosure, all new fans, and two little 5 sata drive tray to put in the new case. For a raid that holds ten drives I can easily spend $1000 just for the enclosure. Plus I have my LTO4 in the Cyclone, and an enclosure for the LTO can cost $250. So half the total cost of the Cyclone I needed to spend anyway just on enclosures.

Plus I save a lot of space. Instead of Mac Pro, Raid, LTO, and Cyclone, I just have Mac Pro and Cyclone. Also uses less power than all combined.

When I ran the numbers, it was a no-brainer.


Dusty
 
Anson Fogel said:
Jordan,

... just getting a 2nd MP Tower, but that probably is another thread altogether.
Click to expand...

That's the reason Apple don't want to put many slots in to force us to by more machines. It really annoys me as it negates their initial philosophy anybody remember this TVC?
http://www.youtube.com/watch?v=R706isyDrqI

A PCI-E expansion chassis gives us the freedom we need.

I don't think that having multiple cards in the one chassis connected via a singe lane Express slot would be a problem cause all cards are controlled via the PCI-E switch. It is in theory suppose to delineate between different streams of data and allow each card to communicate with the mother board separately. This normally results in latency but that would be a performance hit I am happy to take for the ability to play back 4k realtime off a mini SAS Raid 5 and monitor out to HD SDI anywhere any time if need be. Then again this is all speculations as I haven't played with one yet.

Though I normally recommend to tailor a system to a specific need. Download and back back on a station different to the one your grading, look creating, rough assembling or compressing offline on.
 
I've played with the iStorage Pro unit. It is awesome. BUT.... their current connection to the host system is via a 4X PCIe bridge. That is half the maximum bandwidth that the RED Rocket can potentially use. REalistically, 4X is going to be just fine most of the time for the Rocket. But you're not going to stick a whole lot of anything else in that unit when the connection to the host system becomes a bottleneck. If you have both a Rocket and a performance RAID adapter in that same unit operating in unison, they have both the latency from the expander system as well as competition for that same narrow data pipe.
 
Hey guys,
Do you have a preferred cable/connector to run between the MBP's Express34 slot and the iStorage ports?

I can see it needs either:
PCI express X4 MacBook host connector
PCI express X8 MacPro host connector
to connect to the host computer.

What are you using for this? I'm in more need of the PCIE x4 solution in order to connect to the iStoragePro, but don't know what I'm looking for. Any help would be great!

Thanks,
Steffen
 
You must log in or register to reply here.
Back
Top