Divinci Resolve With Cubix GPU Xpander

[Jesse Korosi]

I am just curious if anyone out there has got there hands on the Cubix GPU-Xpander to use along with Divinci Resolve yet?

Will this speed up renders as well or just the playback/realtime performance of your color grading experience?

I am currently running Divinci on a MacPro with the Quadro FX 4800 along with a RedRocket. But I am curious if I would get better performance for specifically Rendering if I were to add another rocket ... or two, or another Quadro FX card?

Thanks in advance!

 

[Steve Johnson]

Jesse,

I spoke to a company here in the UK called jigsaw24 about this and got talking with one of the techies. I asked how much to full 4K playback etc you could get if you added the expander box with 3 Quadro 4000 cards as per the spec on the website.

He said that they where talking directly on a weekly basis with the guess test Davinci for Mac and with the expander box and 3 Quadro 4000 cards, they where getting full 4K half debay with 24 nodes, so I would get you could get 4K Full debay with 12 nodes.

You might want to get that checked independently but thats what I was told.

Hope that helps.

Steve

 

[Jesse Korosi]

Hey Steve,
Thanks for the response!

That is awesome. But I am not getting a 4K monitor ANYTIME soon. haha My main concern was whether or not I could speed up rendering specifically if I had the Cubix.

One of my co-workers was just told by a Divinci Rep today that we would be best served to go with a 12 Core Mac if we were to mainly use it for transcoding.

"The GPU will make it faster but when working with compressed formats the CPU
plays a greater role.

So if you were going DPX to DPX I would say yes as many GPU's as possible
but if you had to pick between CPU or GPU additions with h.264 etc I would
say go 12 core CPU"

 

[Nick Shaw]

…they where getting full 4K half debay with 24 nodes, so I would get you could get 4K Full debay with 12 nodes.

I don't think that logic works. Apart from anything else, 4K is four times as many pixels as 2K, not twice as many.

 

[Jeff Kilgroe]

I don't think that logic works. Apart from anything else, 4K is four times as many pixels as 2K, not twice as many.

Correct. 4X the amount of calculations for the demosaic / debayer operations to go from half to full debayer. Not sure how that would translate to overall performance as there's a lot of other things going on as well. But I would say that it's going to be less than just half as fast.

 

[Chris Parker]

also, as i understand, the GPU expansion helps only with applying color fx or nodes on an uncompressed image (eg. 2K DPX). But if your source material is compressed (eg. QT) then it is CPU power you need more than GPU power.

and that the gpu expander won't work any magic on render times.

if you are needing to have more nodes applied to 2K DPX files and maintain realtime playback and monitoring, this is where the multi GPU comes in.

is this correct?

 

[Dustin Cross]

Decklink is also talking with Cyclone to confirm the Cyclone 600-2707 works with Resolve. I looked at Cubix and Cyclone and went with the Cyclone for my DIT cart and have been extremely happy with it. PCIe Expanders are great. Haven't tried Resolve yet.


Dusty

 

[Jeff Kilgroe]

Any further news on the Cyclone and the ATTO 6Gbps SAS cards? I would want the ATTO R680 to work in Slot-4 of the Mac tower, so not in the expander itself, although I may throw an R308 3Gbps card in the expander and do an internal RAID, maybe with LTO, there.

 

[Dustin Cross]

Jeff,

Last I heard, Atto had a Cyclone and was working on it.

I really like having my R30F in my Cyclone with 10 drives connected and an LTO4. I am thinking about getting a half height LTO4 and adding three more drives. The R30F with 10 drives in RAID 6 does 500MB/s no problem. That is enough speed for now.

Can't wait to be able to use the R60F and get even more speed out of my raid. Should be able to get closer to 1GB/s with 13 drives then.


Dusty

 

[Steve Johnson]

Hey Steve,
Thanks for the response!

That is awesome. But I am not getting a 4K monitor ANYTIME soon. haha My main concern was whether or not I could speed up rendering specifically if I had the Cubix.

One of my co-workers was just told by a Divinci Rep today that we would be best served to go with a 12 Core Mac if we were to mainly use it for transcoding.

"The GPU will make it faster but when working with compressed formats the CPU
plays a greater role.

So if you were going DPX to DPX I would say yes as many GPU's as possible
but if you had to pick between CPU or GPU additions with h.264 etc I would
say go 12 core CPU"

Hi Jesse

No problems - hope it was some help.

Steve

 

[Steve Johnson]

I don't think that logic works. Apart from anything else, 4K is four times as many pixels as 2K, not twice as many.

Hi Nick

Yes i would agree with you on that logic. That is what I thought at first but that's what these tech guys told me. Never believe a tech guy! ;-)

Steve

 

[Steve Johnson]

Correct. 4X the amount of calculations for the demosaic / debayer operations to go from half to full debayer. Not sure how that would translate to overall performance as there's a lot of other things going on as well. But I would say that it's going to be less than just half as fast.

It would be interesting to see on a system that actually is running using the expansion box and the extra 4000's to see exactly what you can get out of them in terms of what resolution vs what debayer vs how many nodes!

 

[Steve Johnson]

also, as i understand, the GPU expansion helps only with applying color fx or nodes on an uncompressed image (eg. 2K DPX). But if your source material is compressed (eg. QT) then it is CPU power you need more than GPU power.

and that the gpu expander won't work any magic on render times.

if you are needing to have more nodes applied to 2K DPX files and maintain realtime playback and monitoring, this is where the multi GPU comes in.

is this correct?

Chris,
So does that not make the case to work in an uncompressed format on Davinci for Mac? So by that rational, correct me if I am wrong, if you work in r3d files, which are not compressed (e.g. QT) is that not better as you would have all the RAW data, so therefore the GPU expander would work. Is that right?

 

[Stephen Lovett]

Hi All,

Dustin, I'm pretty much heading in the same direction.

btw Steve, R3D files are compressed, and then there is the debayer..., which leads to the heart of my question.

So I'm looking at a similar product that hasn't been released yet. It provides PCIe expander capability and up to 10 drives in the same enclosure.

I've got a Rocket, Quadro 4000, a GTX 285, a 120, plus the BMD 3G card, ATTO raid card etc., pretty much what most of us will use.

The $64,000 question that I don't have an answer to is what is the bottleneck? Is it debayer / rocket, CPU, or GPU? Right now I don't have a rocket in the Resolve box, as a fast raid is pretty key, the rocket is in the Storm/Smoke box.

In an ideal world with multiple rocket support, and now multiple GPU support, I'm a little perplexed on what the right combination is.

My gut tells me one rocket, two GPU cards, plus a good GUI card and a fast raid, plus a SAS HBA for e-sata fan out and LTO connect, but I don't really have a way to test this until I get the PCIe expander.

As this is going to be a dedicated finishing box, I can live without the HBA if another Rocket or GPU helps the mix.

The key question is specifically for Resolve what combo of Rocket / GPU is the best balance.

Thanks for all the input folks, it this kind of information exchange that makes this board super valuable.

Steve
Sure wishing that Apple would bring out a competitive motherboard...

 

[Steve Johnson]

btw Steve, R3D files are compressed, and then there is the debayer..., which leads to the heart of my question.

Fair point - I think the wine here is flowing too well - it is almost 10pm on a Friday night in Glasgow!

My gut tells me one rocket, two GPU cards, plus a good GUI card and a fast raid, plus a SAS HBA for e-sata fan out and LTO connect, but I don't really have a way to test this until I get the PCIe expander.

As this is going to be a dedicated finishing box, I can live without the HBA if another Rocket or GPU helps the mix.

The key question is specifically for Resolve what combo of Rocket / GPU is the best balance.

This is what I was trying to get the answer too - the other option is to spend $100k on a Quantel Pablo!!!!

Sure wishing that Apple would bring out a competitive motherboard...

Not sure that's going to happen anytime soon ;-)

I think the question should now be, what is the best solution to do a full DI at the highest resolution? Granted, the first question is: 1. What are you delivering too ( see, I did learn something at REDucation!)

But I think where possible you should try and use the best solutions available.

Listening to the latest RED Centre podcast with Mark from OffHollywood, i would agree that for DI work, we need to have the same as a great distribution model for displaying 4K - an affordable 4K display / projectors...

Just my thoughts...

 

[Dustin Cross]

Stephen,

I put my Cyclone in a new 4u enclosure that allows me to put 13 drives and a half height LTO4 drive. Connect all that to an Atto R30F and everything is great.

That leave three X16 slots and one X8 slot open in the Cyclone.

I am pretty sure Resolve will work great with the Cyclone, but I haven't purchased Resolve yet to verify.


Dusty

 

[darin wooldridge]

I've been beating this same horse. Here is my take on the situation.

The red debayer process is done by the mac pro cpu. period.. Hit play and monitor you cpu performance.. It kills it.
Faster cpu's paired with a redrocket or two will help for high quality playback. The gpu cards are used for color correction processing.
The resolve works best and is designed around dpx frames.
For red playback the bottleneck is cpu paired with 4k resolution.
To do what you are asking you will need 2 red rockets and 2 gpu cards and may still need to preview at a lower debayer setting. Half or lower.
I believe the linux system will give you real time playback at full quality.

My big question. What is best for the final outcome?

1. Grading from a compressed raw / .r3d file.
2. render at full quality debayer to a fat 10, 12 or even 16bit file to start grading.


Just found this posted by Peter from black magic.. He does a much better job explaining the bottleneck.

http://reduser.net/forum/showthread.php?t=53826&page=2





NOTE: The comments above are strictly mine, and may not necessarily represent those of my employers.




Darin Wooldridge
Colorist / Technical Strategist
818-653-3918-cell
dwooldridge@mac.com
check me out at http://www.facebook.com/pages/Davinci-Resolve-Colorist/117363011609028?ref=....

 

[Chris Reynolds]

Hi All

Going thought the whole Cubix Expander My self. The interesting thing I found out was the GPU-Xpander Desktop 2 (what Blackmagic recommend). Has only a Maximum throughput of 40Gbps. Where as the Cubix GPU-Xpander Desktop 4 uses the full 16 lanes. Which to me if you a fully populating the expansion box you would want all 16 lanes i.e. 3 Quadro 4000 and a rocket card.

Cheers, Chris

 

[Tim Sutherland]

Cubix is also coming out with a GPU xpander 2 rackmount version, which will use the full 80Gbps with the 16 lane host card, and allows 2 double height or 4 single height 16 lane cards. I think this is the route I might go because I already have rackmount space for it on my cart.

Tim

 

[Jeff Kilgroe]

For the Cubix Xpanders, the only two that appear to uspport the full PCIe 2.0 X16 bandwidth are the Pro 2 model and the Xpander 80 model. The Desktop 80 model appears to be available now. I think the Xpander 80 would be the best choice as it gives four slots. Then you could install 2 GPUs and a Rocket card into it. That might be a bit much, though. Maybe two GPUs and the Decklink card? I don't know... I'm debating between that and the Cyclone. But will hold off a bit. Going to get the new Epic cameras here and hopefully by the time Intel gets closer to shipping the new workstation chipsets and CPUs (May / June release it seems), we should have a better idea of what to expect from an updated Mac Pro. All I can say is that I don't see how Apple can only put 4 slots in the next Mac Pro. They would be seriously crippling the system and bandwidth afforded by the upcoming chipset. As it is now, at least all 40 lanes are accounted for, even though they didn't expand to 64 via the dual chipset cascade option.

I guess we'll see...