Need advise for Nvidia MacPro

[Marc Berger]

Since its possible to put other Nvidia cards then GTX285 and Quadro 4000 into the Macpro, It would be great you could share experiences or thoughts about it. Right now I have a ATI HD5770 in slot 1, RedRocket in Slot 2, and a pcie e-sata Raid controller in slot 4.
How can I add a NVIDIA card for Cuda/Resolve and PPR? And which one is good for it, GTX470/480.....? Or 2 of them instead one ATI?
Thank you a lot for your input!
Marc

 

[Jeff Kilgroe]

Some quick thoughts....

ATI 5770 is a double-width card and slot-1 is your only double-width slot. It also uses both available PCIe power connectors.

Current nVidia drivers fully support all GTX cards from the 200 series thru the 500 series on OSX. These are the drivers downloadable from nvidia.com, not the ones included in OSX. This driver set is incompatible with the current 10.7.4 beta seeds, so don't install 10.7.4 anytime soon. And only compatible with 10.7.3, which is actually running well on the Mac Pro systems (10.7.0 and 10.7.1 were flakey on the Pro).

Although support is there, unless the card has an Apple-compliant EFI, you will not be able to option-Boot to list of bootable devices, see the apple-logo boot screen or perform other tasks outside the OS as the system won't be able display anything from the non-EFI video card in those situations.

GTX470/480 are also double-width cards and require two PCIe power connectors.

Premiere requires CUDA acceleration on the displays where Premiere is active in order to be properly accelerated.

Resolve requires dedicated CUDA GPUs that are not being used for display. So if you only have one CUDA card in your system, you will have to juggle monitor cables between cards when you go from Premiere to Resolve and back.

Resolve now supports OpenCL and is partially accelerated by the ATI5770. I personally have not worked with Resolve on ATI GPUs to see how the acceleration works, but unlike their CUDA implementation, I believe you don't need a separate GUI GPU.

You can install a double-width GPU card in slots 2 and 3 of the Mac Pro, but you end up sacrificing the slot above, obviously. If you install more cards than you have available PCIe power connectors for, you will need to bring in external power or pull power from somewhere else.

How are you monitoring from Resolve? It's another topic entirely, but it's kinda hard to do proper grading when you don't have a proper output card feeding a calibrated display.

Many people who are serious about running Resolve have moved to using a PCIe expander box. Then you can install a couple lower-cost CUDA cards in that unit. For about US $3200, you can have a Cubix GPU-Xpander dual slot and put two GTX580 cards in it with that connected to your second slot in the Mac Pro. Unless the system is mostly dedicated to Resolve, people are either sticking with their ATI card for primary GPU or if they need CUDA on the primary card, the top choice is the Quadro 4000 Mac edition.

Another possibility is to install two Quadro 4000 (Mac) cards in your tower. It's cheaper than using the Expander and not as powerful for accelerating Resolve, but it still does rather well. This is actually the setup I'm running currently as I've sold my PCIe expander and some other things recently. My current Resolve system is an '09 Mac Pro, 8-core 2.93GHz with 24GB RAM, Quadro 4000 card in slot-1 and slot-4, RED Rocket in Slot-2, DeckLink HD Extreme in Slot-3. I'm most likely moving all my Resolve work over to Windows in the coming months anyway. My 6-core i73960X workstation I built a few months back, handles Resolve beautifully. My only complaint about the current Mac Pro setup I have is I could really use one more slot to put the SAS/SATA card back in. I have to do backups over the network. Not a big deal, I'm using Time Machine for backups on this system to a NAS volume on a server where it gets written to LTO tape on a semi-regular basis. ...Really wish the system had 10Ge in this case too...

When I had the Cubix expander box, I was using the 4-slot unit. That was installed in slot-2 and contained two Quadro 4000 cards. I also still had a Quadro 4000 in slot-1 in the tower. Slot 3 and 4 in the tower had my ATTO 6Gbps SAS/SATA card and the Decklink HD 3D+ card. RED Rocket was in the expander with the two GPUs and I also had a CalDigit USB3 + eSATA card in the expander as well. I figured the expander would bottleneck itself between two GPUs and a Rocket, but it performed great.

If you keep everything within the tower and don't want to spend the money for the two quadro cards, your next best bet is something like a GTX480 or GTX560 in slot-1 to accelerate Resolve and then pick up an older GT120 Mac edition card, which takes a single slot and no extra power connector, to use as your display GUI card. If you want to accelerate Premiere, swap the display cable(s) from the GT120 to the GTX card... :/ Personally, I've been there, done that. It was a pain and didn't always work. There's also a single-slot, low-power GTX520 card out there, but not specifically for the Mac, so no EFI, meaning it will have the restrictions I listed above.

 

[Mike P.]

Hey Jeff, kind of niche question, but have you tried connecting two cards in SLI (and leaving them physically connected) even though you don't use them as such in Resolve or CS5.5?

Let me explain; say I was using a hackintosh machine and wanted to have GPU-accelerated MPE (only uses one GPU) and Resolve (needs two+ GPUs but as far as I know, doesn't use SLI... Plus I don't think OSX can use SLI either), but then on the same machine I dual boot into Windows for some gaming (which takes advantage of SLI)... You think it would cause any problems? I'd just hate to have to keep opening the tower and pulling off the SLI connector every time I want to switch platforms. Similarly, if they're connected via SLI in windows, does it not play nice with the windows version of Resolve?

 

[Terry VerHaar]

Something of an aside - Jeff, your encyclopedic knowledge of all things technical is nothing short of amazing!

Now, back to our regal;ar scheduled program. :smiley:

 

[Marc Berger]

Hi Jeff, thanks a lot for your very detailed input!!! It helps me a lot. When I heard the news about the newly enabled cards I thought very naive: " Wow, great, now I can just open my Tower and put any Nvidia card", but I had a instant feeling there is more to know about it. You´re very generous with your knowledge (I agree with Terry. Amazing).
Your own actual solution looks best to me.
Would be great to have a cubic expander solution with a thunderbolt connection, and then put the graphic cards and Red Rocket in there and use it with Macpro (with thunderbolt) and Macbook pro...but I guess it will take a while until all needed parts are available.
Cheers,
Marc

 

[Jon Jones]

Yeah Jeff is our foremost expert in this arena. His experiential knowledge blows most of ours out of the water. I can hardly imagine the many sleepless nights and rendering heartaches Jeff has experienced throughout the years, lol.

 

[Jarek Zabczynski]

So then additional cards in an expander won't help in Premiere?

 

[Jeff Kilgroe]

Hey Jeff, kind of niche question, but have you tried connecting two cards in SLI (and leaving them physically connected) even though you don't use them as such in Resolve or CS5.5?

Let me explain; say I was using a hackintosh machine and wanted to have GPU-accelerated MPE (only uses one GPU) and Resolve (needs two+ GPUs but as far as I know, doesn't use SLI... Plus I don't think OSX can use SLI either), but then on the same machine I dual boot into Windows for some gaming (which takes advantage of SLI)... You think it would cause any problems? I'd just hate to have to keep opening the tower and pulling off the SLI connector every time I want to switch platforms. Similarly, if they're connected via SLI in windows, does it not play nice with the windows version of Resolve?

I've never tried it personally, but it should work just fine.

Hi Jeff, thanks a lot for your very detailed input!!! It helps me a lot. When I heard the news about the newly enabled cards I thought very naive: " Wow, great, now I can just open my Tower and put any Nvidia card", but I had a instant feeling there is more to know about it. You´re very generous with your knowledge (I agree with Terry. Amazing).
Your own actual solution looks best to me.
Would be great to have a cubic expander solution with a thunderbolt connection, and then put the graphic cards and Red Rocket in there and use it with Macpro (with thunderbolt) and Macbook pro...but I guess it will take a while until all needed parts are available.
Cheers,
Marc

Yeah, it's never as simple as we'd like. Even in PC land where we can essentially choose whatever GPUs we want, there are still issues with what the hardware can actually handle and what components will get along with others. As for the expander solution with Thunderbolt, that's an option coming soon -- we'll have to hope for Thunderbolt ports on the new Mac Pro, which I've been told by a couple reliable sources that it's all coming. The one issue we have with Thunderbolt though, is it's not as fast as a lot of people keep thinking it is. Yes, it's fast -- 20Gbps in a single port via two 10Gbps channels. But there are some restrictions -- current device headers only allow for 10Gbps to be used per device with the second channel available for pass-thru, if the peripheral maker decides to implement that aspect of it. More capability is coming and an effective speed doubling with the shift to PCIe 3.0 and optical cables -- probably later this year or early next year for actual release.

As it currently stands, the 10Gbps of bandwidth available to a Thunderbolt PCIe expander, like the one from Magma that was demoed at IBC with the Red Rocket, equates to two lanes of PCIe 2.0, or 4 lanes of PCIe 1.x spec. It allows the RED Rocket to run at half it's maximum throughput. So it's not like you're going to cram a bunch of other stuff into a Thunderbolt expander.

Yeah Jeff is our foremost expert in this arena. His experiential knowledge blows most of ours out of the water. I can hardly imagine the many sleepless nights and rendering heartaches Jeff has experienced throughout the years, lol.

Sleep? Around here it's just "night" or that time of day when my phone doesn't usually ring.

So then additional cards in an expander won't help in Premiere?

Nope.

Premiere only uses one GPU for acceleration and it has to be the primary GPU on which Premiere is working / displaying viewer windows. In most situations, a GPU like the GTX480 or GTX560 and up is plenty good with what MPE uses it for. And the GPU is not going to be your bottleneck.

 

[Christopher Barrett]

Nope.

Premiere only uses one GPU for acceleration and it has to be the primary GPU on which Premiere is working / displaying viewer windows. In most situations, a GPU like the GTX480 or GTX560 and up is plenty good with what MPE uses it for. And the GPU is not going to be your bottleneck.

Oh, drag... I've got 2 4000's in the expander and run the displays off of a 5870. I wondered why I wasn't getting better Premier performance. So I should ditch the 5870 and get another 4000, eh?

 

[Jeff Kilgroe]

Oh, drag... I've got 2 4000's in the expander and run the displays off of a 5870. I wondered why I wasn't getting better Premier performance. So I should ditch the 5870 and get another 4000, eh?

Pull the ATI card and replace it with one of your 4000's. See how that does for you with everything before you spend the $$$ on yet another quadro. I suppose it depends on how hard you're pushing the system in Resolve as to whether you really need two of those Quadros in that expander. No other app is going to use them. If you're going to buy another card, you would probably do better to put a GTX580 in the expander and use it for the CUDA acceleration in Resolve instead.

The only real issue with using non Mac specific GPUs for Resolve in this case is that the current nVidia drivers work and make it easy to use. No guaranty that will continue with future OSX and nVidia driver release combinations.

 

[Josh Negrin]

I have an NVidia gtx 580 in my 2010 mac pro running the latest version of lion. I can speak from experience in how awesome it is to playback 4k footage in realtime in premiere pro cs 5.5. Also, I switched over to premiere pro for editing a 2 hour show every week. I haven't comepletely ditched fcp yet, but I'm getting there.

-josh

 

[Jon Jones]

Jeff, I know that we have discussed this somewhat before, but would it be possible for me to install a Quadro 6000 in my MAC PRO, and utilize the 6GB of RAM and the CUDA cores for Premiere as well? I really want your answer to be yes I have a secret formula that will work, HAHAHAHA! But I'll take whatever, lol

 

[Jeff Kilgroe]

Quadro 6000 won't work inside the tower, you would have to put it into a PCI expander. Or at least I'm pretty sure that's the situation. Also not sure about driver support for the 6000 on the Mac. IMO, save the money and go with the GTX580, it screams. If you think you still need more (and the law of diminishing returns kicks in here pretty quickly) you can add a second one or choose the GTX590 instead. The benefits to running the Quadro cards is not raw performance, they offer higher precision for OpenGL performance and specific functions, gunlock support and a few other bells and whistles that are not going to do anything for you on OSX at this time, especially if it's a secondary card being used to augment CUDA.

 

[Brian Merlen]

I saw some guy on ebay is selling hacked 6000's to work in mac pros, someone mentioned it in another thread...some where....i am on set, don't have time to find a link...

personally i am weary of hacked stuff, whether its hackentosh or hacking a pc only card into a mac, you have no support from apple/nvidia by hacking so personally i wouldn't risk it...however some obviously do...

 

[Jon Jones]

Yeah I have been reading about others hacking their command script, and soldering something or rather, to use the higher end cards. Not sure I'm ready to fudge up my investment at this particular moment. Jeff, I think that I may go with your recommendations however on the dual GTX580's. You are talking about the 3GB cards arent you? Also I have a cubix expansion bay coming my way sometime in the next month, can I put them inside of that along with 2 more rockets?

 

[Jeff Kilgroe]

Yeah I have been reading about others hacking their command script, and soldering something or rather, to use the higher end cards. Not sure I'm ready to fudge up my investment at this particular moment. Jeff, I think that I may go with your recommendations however on the dual GTX580's. You are talking about the 3GB cards arent you? Also I have a cubix expansion bay coming my way sometime in the next month, can I put them inside of that along with 2 more rockets?

Which Cubix are you getting? Two GTX580s and two Rockets would probably overload that PCI slot anyway. There comes a point where you may do better to build more computers than trying to stuff all this crap into one. Your drives, CPUs, memory, etc.. can only handle so much and work so fast. Keep in mind the GTX580s are double-width cards and each take two PCIe 2x8 power connectors. The standard Cubix GPU-XPander boxes will only hold two of the GTX580 and nothing else as they are only sold with two double-width slots or 4 single-width slots.

 

[Jon Jones]

Hmmm, I have an idear, what if I put another MAC PRO in my bigfoot cart and connected them through the 32TB Raid array? Then I can split up jobs between the two, and is there a way to share the red rockets between units?

 

[Jeff Kilgroe]

Hmmm, I have an idear, what if I put another MAC PRO in my bigfoot cart and connected them through the 32TB Raid array? Then I can split up jobs between the two, and is there a way to share the red rockets between units?

Hmmm... What are you trying to accomplish? Is it faster transcodes? Or what specifically? You can connect both systems to the same 32TB RAID, given the proper connecting hardware. Basically you have to set up a small SAN.

No way to share the Rocket cards between the two, but depending what what you are trying to make happen, you may or may not do better to have them in one box. Actually that's not really true about the sharing, you can use the command-line renderer (REDLINE) and interface it with a network render manager software like ButterflyNet, Chalice, etc.. So you can help automate splitting transcodes or renders across multiple systems and each will use whatever Rockets installed for the task presented to them. Honestly though, that's really a pain unless you have a ton of systems in a render farm. For 2 or 3, you're probably best off to just come up with a system where you divide up your transcode queues and dump them to the same storage volume.

The biggest thing to keep in mind here is that no matter how much hardware you try to pack into a system or how much money you throw at this, there is only so fast you can make it all work. The more exotic or elaborate you try to be with your hardware configurations, the greater the risk of things not working as they should. ...And also more work and $$$ involved to fix things when something breaks.

 

[Jon Jones]

Ahhh, yes well your knowledge on these subjects is dizzying, I just woke up man, settle down, lol. No but for real, I understand what you are saying. Point taken. I will just start with dual 580's, I'll put one in the cubix bay, and one in the tower, and then see where I am at.

 

[Walter R]

Hi,
I see a few folks talking about a GTX 580 in a Mac Pro tower in this thread. Is that even viable given the power requirements or are you guys adding an additional power supply?