THE beast 32 core HACKINTOSH is ready.....CS6 vs REDROCKET

[Joseph S.]

If you can find these download links, I would be more than happy to test it out. I don't know where to look for them.

Joe

 

[Jeremiah P]

Hey Joe,
Thanks again! Since I currently can't run Resolve on my dated machine I can't check this stuff out....but here is a link to the project file. Candle Test Project. And here is a link to the footage I think it needs to be reconnected to. Video Clips. If you could write down the framerate for the various examples given by Sasha on the CreativeCow forum that would be great! I would also love to know what sort of performance can be had on GTX 680s if you know anyone...since those are more affordable and available. Thanks again!

 

[Gabriel Scindian]

This is an AMAZING Peice of TECH! Please give specs. I need something similar

the beast has been tamed ...:devil:
many hours and much suffering, it works now.
sandy SUPERMICRO mainboard X9DRi-LN4F+ with e2690 dual cpu xeon....

osx 10.7.3, 11.3.0 kernel with support for ivy bridge,
32 cores with steep speed enabled.
Support for 4 gpu tested. for now in testing fx1800 and 9800gx.
support should take 2 gtx 590. In the coming days we will test
4 port lan enabled.


Joe Safai is doing some testing over the next few days will keep you updated.
If anyone is interested in some tests with R3D files happy to oblige.

CS6 5k for now all in real time full frame at full resolution ..... without redrocket.:emote_22_yikes:
Geekbech above 30000 points.


Thanks to Joe for your patience.

 

[Adam Welch]

That is pretty freaking badass!!!

 

[Henrik Cednert]

Joseph and Lugi. If you built this system today, would you change anything and do something different? How have the systems performed over the months you have been using it? Are there any annoyances that you constantly run into? Have you guys gone up to 10.8.2?

Thanks

 

[Jon Thomasberg]

Guys (Luigi, Jeff, Joe),

I see you used a Supermicro X9DRI-LN4F+ mobo for your build. Given the new boards that have come out, especially the ones with LSI SAS onboard controllers and USB3, would you still use that same board vs say a X9DAX-7TF? Trying to spec a new build now and would appreciate you thoughts and input.

Also, on the storage side, looking at raid 10 (4) Samsung 840 Pro SSDs for the OS+apps; (1) SSD for swap and then the 8bayRAID SAS using WD RE4 7200 4TB (WD4001FYYG) drives for R3Ds and export. Thoughts?

Perhaps a better question is: If you were spec'ing-out a build today, what components would you use? (+1 Henrik)

TIA

 

[luigivaltulini]

Guys (Luigi, Jeff, Joe),

I see you used a Supermicro X9DRI-LN4F+ mobo for your build. Given the new boards that have come out, especially the ones with LSI SAS onboard controllers and USB3, would you still use that same board vs say a X9DAX-7TF? Trying to spec a new build now and would appreciate you thoughts and input.

Also, on the storage side, looking at raid 10 (4) Samsung 840 Pro SSDs for the OS+apps; (1) SSD for swap and then the 8bayRAID SAS using WD RE4 7200 4TB (WD4001FYYG) drives for R3Ds and export. Thoughts?

Perhaps a better question is: If you were spec'ing-out a build today, what components would you use? (+1 Henrik)

TIA

Ciao Guysss...
good question, I prefer to take a PC already 'done, as hpz 800/820 or Dell T7600. I believe that this field can answer Jeff, he be the man who can give you a perfect answer as to which piece is better than another, and on their speed and construction.

I personally am waiting for the new Mac Pro, even though I have lost confidence. But I can also tell you that I'm going to a Dell or HP z820 T7600.
I have not been bought because i have problemsss of Turbo Boost on Sandy Bridge Chipset C602 and C606 and still some problems on boot.

My next setup, if you can be of help:
2 x 6 core HPZ 820 or Dell T7600
64 gb ram
1 gtx 690
1 8800 gt or 1 ati 5770 GUI
2 x redrocket
atto H680


Luigi

 

[Henrik Cednert]

Aaah. Gotcha. You could buy a prebuilt supermicro workstation too. I like the HP Z800/820 but I personally prefer the bigger form factor and expansion possibilities of a supermicro system. But you're saying that for now you basically abandoned the Hackintosh path due to issues that was't possible to solve?

Cheers and thanks

Ciao Guysss...
good question, I prefer to take a PC already 'done, as hpz 800/820 or Dell T7600. I believe that this field can answer Jeff, he be the man who can give you a perfect answer as to which piece is better than another, and on their speed and construction.

I personally am waiting for the new Mac Pro, even though I have lost confidence. But I can also tell you that I'm going to a Dell or HP z820 T7600.
I have not been bought because i have problemsss of Turbo Boost on Sandy Bridge Chipset C602 and C606 and still some problems on boot.

My next setup, if you can be of help:
2 x 6 core HPZ 820 or Dell T7600
64 gb ram
1 gtx 690
1 8800 gt or 1 ati 5770 GUI
2 x redrocket
atto H680


Luigi

 

[luigivaltulini]

Aaah. Gotcha. You could buy a prebuilt supermicro workstation too. I like the HP Z800/820 but I personally prefer the bigger form factor and expansion possibilities of a supermicro system. But you're saying that for now you basically abandoned the Hackintosh path due to issues that was't possible to solve?

Cheers and thanks

ahah Henrik,
no I have not abandoned are just waiting
Supermicro... yes i like this , only this 7047GR-TRF.

 

[Jeff Kilgroe]

My two Xeon E5-2600 workstations are the Z820 and a self-built with the SuperMicro X9DAi motherboard. Like this generation of the Xeon platform, both systems are far from perfect.

If I were to build another one, I would definitely consider many of the newer motherboards on the market. Starting with the new EVGA SR-X.

The Z820 is great for a production workstation, but damn, they made some curious and obviously hasty design choices. 99% of which I'm sure has to do with how they decided to shoehorn the new platform into the existing Z800 form factor. The PCIe slot layout is crap. Then again, it's crap on most every board I've seen for this generation. I have yet to see any PCIe configuration on any of these workstations that allows for use of all PCIe slots and lanes. It's just physically impossible with the layouts these companies are coming up with when we consider that most all GPU and compute cards require two slot spaces. IMO, Dell has the best PCIe layout on their T7600, or whatever it's called, flagship E5-2600 workstation. But they were really late to market with it and it hasn't garnered the best reviews for performance or reliability. HP is still leading the pack with the Z820 as far as this generation goes. Solid product and great support. I just wish they would've taken some more time with the product and really thought through the design, rather than making it fit into the box they already had. Some extra spacing for the PCIe slots, shifting the CPUs and RAM upward would really help. They could have made the system wider, take an extra 1U of rack space without adding height (rack width) and moved the PSU to the side instead of on top. This could have allowed for a larger PSU, which the system NEEDS!, and for some additional space to augment airflow and allow us to actually install some of the larger GPUs that need some extra height extension off the motherboard. Their airflow guide that snaps over the PCIe card region is a fine idea, but it encroaches into the card region. There are several GPU cards I wanted to try in this system and could not use due to the airflow guide being too restrictive.

 

[Nicos Ambatzis]

Hi Jeff,

I would be glad to know your opinion of the X9DRG-QF motherboard. What is that you don't like and you don't mention it???? Of course it doesn't support USB 3 into motherboard and has no sound on it. Nevertheless, why Blackmagic propose it as basic workstation with the HD Z820????? You must know something we don't. Thanks Jeff.

 

[Henrik Cednert]

Hi Jeff

Actually the EVGA SR-X is EOL, or rather end of sales, and they won't produce any more of them. From what I've been reading EVGA doesn't have that good of a track record when it comes to keeping EOL products alive when it comes to BIOS etc, purely something I've read an I don't have first hand experience.

The Dell Precision T7600 looks really interesting (owners manual as PDF). "Innovative" to have the CPU's between the PCI-slots! Also room for 8 disks in it and actually a pretty good design with nice touches like a slim optical drive, not that I'd need a optical drive but still a nice touch! =) If we purely look at the slots I believe that the Supermicro X9DRG-QF might be the "winner" with 4 x16 slots. Seems to be on the cost of USB3 though and it's also a proprietary sized board. Personally I don't mind proprietary sized boards since I fancy the supermicro chassis (...even though they for sure could be a bit pimped out like the DELL ones =)).

I agree on all points regarding the HP-system. If I recall correctly and from my experience with the Z800 it has this odd size so that when you lay it on a shelf in a rack it's like 4U and just one hole of the fifth U? It was a while ago I worked with them but I do recall that there was something fishy with them. Same with the Z400. I don't understand why the vendors are so restrictive with the physical size on the flagship workstations. They should all, out of the box, be rack mountable and 4-6U and fill the full width of the rack. WIth some engineering skills they could probably even make them modular so that you could buy like extra sections to it for like 1U, 2U, 3U so that you could grow the box up in the rack (or to the side in tower config). =) Not sure how practical it would be and if it even would be possible but feels like a fairly useable option for us freaks. :shocked: =)

My two Xeon E5-2600 workstations are the Z820 and a self-built with the SuperMicro X9DAi motherboard. Like this generation of the Xeon platform, both systems are far from perfect.

If I were to build another one, I would definitely consider many of the newer motherboards on the market. Starting with the new EVGA SR-X.

The Z820 is great for a production workstation, but damn, they made some curious and obviously hasty design choices. 99% of which I'm sure has to do with how they decided to shoehorn the new platform into the existing Z800 form factor. The PCIe slot layout is crap. Then again, it's crap on most every board I've seen for this generation. I have yet to see any PCIe configuration on any of these workstations that allows for use of all PCIe slots and lanes. It's just physically impossible with the layouts these companies are coming up with when we consider that most all GPU and compute cards require two slot spaces. IMO, Dell has the best PCIe layout on their T7600, or whatever it's called, flagship E5-2600 workstation. But they were really late to market with it and it hasn't garnered the best reviews for performance or reliability. HP is still leading the pack with the Z820 as far as this generation goes. Solid product and great support. I just wish they would've taken some more time with the product and really thought through the design, rather than making it fit into the box they already had. Some extra spacing for the PCIe slots, shifting the CPUs and RAM upward would really help. They could have made the system wider, take an extra 1U of rack space without adding height (rack width) and moved the PSU to the side instead of on top. This could have allowed for a larger PSU, which the system NEEDS!, and for some additional space to augment airflow and allow us to actually install some of the larger GPUs that need some extra height extension off the motherboard. Their airflow guide that snaps over the PCIe card region is a fine idea, but it encroaches into the card region. There are several GPU cards I wanted to try in this system and could not use due to the airflow guide being too restrictive.

 

[Les C.]

My two Xeon E5-2600 workstations are the Z820 and a self-built with the SuperMicro X9DAi motherboard.

As it's 3rd quarter (at least) for the next Xeons, and then some time for the dust to setting on the new motherboard designs, I've been looking at the X9DAi, as that seems the best choice for a SuperMicro motherboard that will fit into an extended ATX case (I refuse to ever buy another SuperMicro case or power supply).

I understand that they no longer use a standard E-ATX, but must drill and tap 4 holes in a non-SuperMicro case.

Did you use a SuperMicro case, or go with another brand?

I have an X8DAE in an Antec case, and have been very happy.

Any specific issues with this board?

 

[Jon Thomasberg]

Guys,

Thanks for your thoughtful replies.

JT

 

[Jeff Kilgroe]

Hi Jeff

Actually the EVGA SR-X is EOL, or rather end of sales, and they won't produce any more of them. From what I've been reading EVGA doesn't have that good of a track record when it comes to keeping EOL products alive when it comes to BIOS etc, purely something I've read an I don't have first hand experience.

Now this is a problem that EVGA's marketing and hardware people really need to sort out. I've seen conflicting reports on this. If it is indeed true and production has stopped, it is most likely due to poor sales, over production, or both. And truth be told, the SR-X was a stinker of a motherboard when released. However, the past couple firmware revisions have rocketed it to stardom amongst enthusiasts and over-clockers. And yes, their track record for support longevity is very sketchy. For now though, supplies seem to be adequate, which may also mean they just stopped production because they don't need any more. And there is something up with the new Xeons coming this year. They are being delayed, but no firm info on why or when we will see them, only wild speculation. There are credible rumors going around about new chipset samples being circulated, when the C60x was supposed to carry the Ivy Bridge Xeons. Intel may have decided to take a breather and get Thunderbolt rolled in, among other things. We can only hope. If that is the case, we will see production cease on a lot of the current E5 motherboards. And manufacturers will probably be rather quiet about it until new products are ready to announce.

The Dell Precision T7600 looks really interesting (owners manual as PDF). "Innovative" to have the CPU's between the PCI-slots! Also room for 8 disks in it and actually a pretty good design with nice touches like a slim optical drive, not that I'd need a optical drive but still a nice touch! =) If we purely look at the slots I believe that the Supermicro X9DRG-QF might be the "winner" with 4 x16 slots. Seems to be on the cost of USB3 though and it's also a proprietary sized board. Personally I don't mind proprietary sized boards since I fancy the supermicro chassis (...even though they for sure could be a bit pimped out like the DELL ones =)).

Nothing really innovative about the Dell design compared to many workstation designs of 15 years ago, back when lots of slots and large CPUs/heatsinks were the norm. Yes, the slimline optical drive is a nice touch, something that happens every now and then amongst workstations and servers, but for some reason not often enough. Of those I know who have purchased the Dell, they love the design, but haven't had good luck with the system as a whole...

I agree on all points regarding the HP-system. If I recall correctly and from my experience with the Z800 it has this odd size so that when you lay it on a shelf in a rack it's like 4U and just one hole of the fifth U? It was a while ago I worked with them but I do recall that there was something fishy with them. Same with the Z400. I don't understand why the vendors are so restrictive with the physical size on the flagship workstations. They should all, out of the box, be rack mountable and 4-6U and fill the full width of the rack. WIth some engineering skills they could probably even make them modular so that you could buy like extra sections to it for like 1U, 2U, 3U so that you could grow the box up in the rack (or to the side in tower config). =) Not sure how practical it would be and if it even would be possible but feels like a fairly useable option for us freaks. :shocked: =)

Yeah, the HP systems essentially take 5U of space when you rack them. Not unusual there, very few full size tower workstations fit evenly into a specific unit space. Most of the EATX and HPTX style cases you buy will require 5U, but won't quite fill it. If you want a system to fit snugly within the designated rack allotment, it seems you need a designated rackmount enclosure. Whatever... The only cases that seem to break that mold are the cheap ones -- where they use the same chassis for both a 4U or 5U rackmount or tower and the only thing that differs when you buy them is which faceplate they slap on it.

The X9DRG-QF motherboard was one I considered. However, it lacked too many things. It's set up mostly for servers or computational systems. Slot arrangement is specifically designed to take several double-wide compute cards. The C602 chipset obviously supports USB3.0, as that's what the majority of the dual E5 boards out there are using, but they chose to go a different route and stack on 10 USB2.0 ports instead -- also a proper configuration for the platform. No audio, no eSATA, no frills... Nice slot arrangement though, would be handy for a Resolve workstation, but you'll still need to add an audio card and probably a fibre channel, 10GE or RAID card to the mix. The proprietary form factor sucks balls, too. All those characteristics somewhat negate the other advantages...


SuperMicro makes boards that fit standard form factors, like the X9DAi. If you find it doesn't fit your case, then your case probably isn't entirely compliant -- also not uncommon. There are different variations of ATX and E-ATX too -- EE-ATX, E-ATXM, HPTX, etc.. Not all motherboard makers follow the specs to the letter, Supermicro usually does pretty good in that department.

The X9DAX series boards, especially the new entries to the series have 6Gbps SAS on an upgraded RAID host. The X9DAX-IFT board trades the SAS RAID for dual 10GE ports on an Intel X540 host. My only real complaint about the X9DAX series boards is the slot spacing. All fine except for the two X16 slots at the bottom. They should have really spaced out the lowest slot so both could have accommodated double-width cards. By my measurements, it still would have fit in the EE-ATX form factor.

 

[Les C.]

SuperMicro makes boards that fit standard form factors, like the X9DAi. If you find it doesn't fit your case, then your case probably isn't entirely compliant -- also not uncommon. There are different variations of ATX and E-ATX too -- EE-ATX, E-ATXM, HPTX, etc.. Not all motherboard makers follow the specs to the letter, Supermicro usually does pretty good in that department.

I read that on NewEgg - about the X9DAI not being standard ATX extended. My X8DAE fits in my Antec case without issues.

For your X9DAI, can you put a 2 slot video card in the X16 on the far left? I'd especially like to use that slot for a video card as there's just an empty space to it's left.

How about the other 2 x16 slots? Any issue with a dual width card in those (where one PCIe slot gets sacrificed)?

BTW - for motherboards, I really do love SuperMicro. Just had zero problem with them - and good tech support on questions.

Stability is the most important thing to me - and I get that from them.

I've been using 3ware PCIe RAID cards (now owned by LSI) - and had very good luck with them also. Never lost any data - and their management software give me what I need in terms of email alerts for problems.

 

[Jeff Kilgroe]

X9DAI fits E-ATX cases just fine. Sadly, many cases do occasionally need an extra hole or two tapped as they often just present you with the bare minimum mount points per the spec. The bottom slot (or far left slot if you look at the board with the ports oriented to the top of your screen) can hold a double-width GPU just fine. You do need to make sure your case provides enough clearance there for proper airflow and so the card will even fit. The other slots work too for that, but you cover up the X8 slots in between. My X9DAi build has a GTX690, works great. I actually have it covering an X8 slot as I don't have all the slots filled and I don't have to worry about the card overheating there.

Supermicro are consistently the most stable motherboards for me as well. I have been using them for years and rarely pick anything else for servers. Workstations I lean toward Supermicro, but it doesn't always work out that way, depends on who can meet my configuration needs. I know lots of people love them, but I tend to shy away from ASUS motherboards. I have had nothing but bad luck over the years with them.

3Ware RAID cards seem fine, I don't have much experience with them myself. Actually I did use them off and on more than a decade ago when I needed fast controllers with a large cache -- they had some of the best cacheing SCSI RAID controllers back then. I tend to stick with Areca, ATTO and HighPoint these days. All of those treat me well and they work consistently across Windows, Mac and Linux. The once great Adaptec and a few others who were big several years ago all seem to have faded away to product obscurity these days.

 

[Les C.]

My X9DAi build has a GTX690, works great.

Thanks for the feedback on the X9DAi!!

I think in an earlier post, you originally had a GTX-580 (maybe with 3GB)?

How has your new GTX-690 (4GB?) benchmarked against it - and with what software?

I'm very curious about CUDA speed.

Sadly would actually like opengl to work also (which nVidia has throttled glReadPixels, etch. in their drivers since the GTX-285's http://www.opengl.org/discussion_bo...-fermi-cards?p=1216119&viewfull=1#post1216119 which were faster than Quadro 6000'S)

Again sadly, it doesn't seem like there's one card from nVidia that does it all. The Quadro cards are lacking in many areas. Too bad there's not a GTX-690 for a higher price with all the opengl calls working without throttling? I'd buy it.

 

[Joel Arvidsson]

My X9DAi build has a GTX690, works great.

Still trying to find what is a bottle neck in my x9dai system since i get so poor performance in davinci resovle with 4k r3d files. I would expect it to be realtime at half res and premium debayer, or atleast good debayer. Is there any good programs to measure where the bottle neck is? I start to think its still the disks where I have atleast 500 read/write.

The system
windows 7 64bit proffesional, sp1
latest nvida drivers.
Supermicro 7047A-T
x9dai mother board
2 x 3.1 GHz 8-core Xeon E5-2687W
8 x 8 GB DDR3 ECC reg 1600 MHz, Samsung
240 GB SSD, Intel 520
EVGA GeForce GTX 680 4 GB, PhysX Cuda
240 GB SSD for os
internal raid with 523mb/s write and 531mb/s read

I must have set some funky settings in resolve. I know I only have one graphic card in it right now. Will add a gtx 690 or something later, but still.


Sorry if im hijacking the thread abit, but Im starting to get bold after all hair pulling.

Does anybody have a working xml from the candle test? please post or pm me a dowload link.

 

[Jeff Kilgroe]

I originally had the EVGA GTX-580 Classified Ultra 3GB, actually tested briefly with two of them. Nice card. It's a better card for Premiere than the 690, since the 690 is two underclocked 680's on a single card. The 690 is a better card in Resolve vs a single overclocked 580, but nothing could top the dual OC'd 580's. I have found that between standard versions of the GTX680 and GTX580 that they're nearly identical for CUDA. Which one has the edge depends on the application and type of operations being run. The 580's are more powerful in a brute-force sort of way. The 680's use more stream processors (CUDA cores) to accomplish the same tasks and at times don't seem as efficient, but they often win the edge with faster DDR5 memory and much faster on and off of the PCIe bus with gen3 -- it shines against the 580 when transaction I/O matters. I have not actually tested the GTX680 4GB. They were slow to market and hard to find. I'm kinda at the point where I won't be testing and building much of anything new until the new Xeons ship and I'll start re-evaluating then with whatever is on the market.

I wish people would lay off the glReadPixels issue. It's a deprecated function and in OpenGL 3.0 and later there are better ways to accomplish that task. No resorting to CUDA or other work-arounds. Yes, that, and several other functions are slower on the newer GeForce cards and it's not just a simple matter of throttling them back. Too many armchair OpenGL developers out there. In the rare instance that you need to read back pixel data you're better off to to use direct buffer access or other more efficient methods. ATI cards have the same issues between their FireGL pro cards and the Radeon series. Slower performance on older functions, different performance in general for OpenGL between consumer and pro oriented cards. Not that it's necessarily a bad thing either way, but the cards are targeted for specific purposes. People complain about performance for certain tasks on the GeForce or Radeon cards. But you don't see as many people complaining the other way -- there are plenty of operations, especially regarding use of the texture pipelines, where the GeForce and Radeon cards run circles around the Quadro or FireGL cards. It's not that the more expensive ones are crippled in that regard. Sure, you can look at it that way, but there are always trade-offs to be made when tuning firmware, drivers, API support, etc.. In the case of glReadPixels and a few other functions that have suffered the same fate, you have to consider the amount of overhead involved to back that function. Personally, I wish they would just pull it from the API entirely! It's only still there to support old OpenGL code and apps that haven't bothered to move on. Apparently it also exists so people can complain about how the non-pro OpenGL cards have been "throttled back". For the most part, you're going to get nearly 60% more performance out of a Quadro in most of your bigger OpenGL apps that rely on legacy code base by running them on a Quadro. Anything that uses hardware stencil buffers, stipple masking, old-style hardware lights, etc.. is going to perform better on the Quadro. Now is it because the GeForce is throttled back? Or is it because the Quadro series has firmware and drivers tuned to provide additional memory buffers and GPU cycles dedicated to the massive amount of overhead required for those functions? Functions that make more sense to do in other ways these days.

The GTX-285 is 3 generations old now for GeForce cards. It wasn't in its hay day until 2008, when OpenGL 3.0 was released and it was the last of the GeForce cards to properly support most of the older OpenGL calls from v2.0 and earlier. It was the flagship of the 200 series and in terms of polygon pushing abilities it was pretty great. It was right up there with the GTX-480 and almost as good as the 580. For CUDA it's a monster, about on par with the 480, but not enough bandwidth to keep up on intense tasks. It's a shame that it wasn't a very long-lived card and support has all but waned for it.

IMO, what I think nVidia should do is drastically cut down on the number of their offerings and ref designs supplied to manufacturers. They should build their top model or two in each series to Quadro specs, complete with the more robust memory controller, ECC support and other features that the Quadros have. And sell them for a more reasonable price because they will be targeting high-end gamers and graphics pros alike with the same cards. Give us the ability to load the properly tuned firmware and drivers to suit our individual needs.

There is no one GPU to rule them all at the moment. Every now and then one does pop up. The Quadro 6000 was almost that when it was released, but when all the overclocked GTX580's hit the market about the same time, that somewhat killed it as king of the hill. When the G120(K20) ships in consumer and Quadro cards here in the coming months, we'll see a nice boost in performance. It's currently available in the Tesla K20X. nVidia hasn't made it clear if it will hit GeForce or Quadro release next. I think probably GeForce as nVidia is really trying to push the Maximus concept with the smaller primary GPU (like the Quadro K5000) paired with additional compute cards in the form of the Tesla.

My ideal nVidia card would be a water-cooled 690 with a 20% + overclock and 8GB. But I'm going to sit tight with what I have and hope the G120 brings us a new series of GeForce cards to play with.