New Workstation Build

[Frank Cueto]

Jeff, how about some cinebench scores on the HPz820?

 

[Jeff Kilgroe]

Jeff, how about some cinebench scores on the HPz820?

Cinebench is starting to show that it's not a great benchmark for these new systems. The test isn't intensive enough or long-running enough to kick in the turbo on the CPUs.

The Z820 system is giving me a render score of 25.31 and 84.98 on the render with the Quadro 6000 GPU.

But like I said, it's not capitalizing on the CPU abilities... I just compared the base numbers on my Z820 for now until I really get to dig into tweaking everything. On my nearly identical build using the SuperMicro motherboard and the same CPUs, I managed a render score of 31.09 in Cinebench. But I had to crank up the minimum CPU in the Windows settings as high as it would go (default is 5%) and I had to run another render in Modo to get all 32 threads firing at full speed then started the Cinebench render test a fraction of a second before the modo render completed! So yeah, I kinda cheated... But it shows Cinebench to be a poor way to benchmark these new systems.

 

[Frank Cueto]

If its not asking too much, how about doing the same cheat on the HP box.

I am Incredibly torn between the HP/Home built and a TOP END BOXX, and I am suffering of paralysis by analysis, if u you know what i mean.

 

[Jeff Kilgroe]

I'll give it a try after I get everything set up on the HP box. I have an issue to iron out with the Vertex 4 SSD I've installed. It's giving me some erratic performance and the system likes to hang and reboot itself one every 2 or 3 times it wakes from sleep. I think that is related to the SSD as it didn't do it when booting from the stock HDD.

FWIW, the SuperMicro build was giving me a 25.10 running the Render test through a few times. So the HP box seems to be tuned well.

Based on the components in the BOXX system for the price I would steer you away from them. They're using an Arima motherboard, from what I can tell, which are usually pretty good, but it's just not at the same level as the SuperMicro X9DAi or their similar offerings or the HP. You can only install half as much RAM and you have fewer PCIe configuration options. It's more comparable to doing a self-build with the ASUS dual E5 motherboard. On that note, I'll say that I'm hearing good things about the ASUS board -- if you don't need all the PCIe slots or the 16 DIMM sockets. It brings some overclocking options to the table as well. Not much as the Xeons are multiplier locked and voltage limited, but there is a little freedom to play there that you don't get with the SuperMicro boards. Not sure what the EVGA SR-X is going to offer. I don't know anyone who has one yet.

 

[Olivier Madar]

Hi Jeff,

I have to choose beetwen Super micro and HP and as you are the only one personn that have both, i ask you that question. I know that the Supermicro is bigger and better but I am afraid of the noise because I can't put the computer behind the wall. So, what is the difference in noise between the two ?


Olivier

 

[Jeff Kilgroe]

I wouldn't say the SuperMicro is better. Only real difference is the PCI slot configuration and since you're assembling yourself, you can place it in as big of a case as you would like. The HP system is very quiet. Comparable to a Mac Pro, the only fan that I audibly hear ramp up under load so far is the one on the Quadro 6000.

The SuperMicro build on my end is noisier at the moment, but I can make it quieter by swapping out fans and whatnot. Not sure what else to say on that, it's as quiet as you can build it. If you want the quietest option, you will want to go liquid cooled.

 

[Clay Glendenning]

Jeff,

Thanks again for being such an incredibly valuable asset to the forums. I bet you are a little tired of the endless questions. I think the timing just so happens to coincide with many of us needing to make large hardware upgrades and studio changes during a very chaotic, for lack of a better word, time in the software and hardware arena for production.

I'll PM you if I need to, before I fill this thread with the build data. I'm building out a Z820, and soloing all of the research. We own a 4-slot double wide cubix with a RR and GTX 570 (space left to add a few things down the road, and we can move around the aforementioned in the new proposed machine). This gets used on a Mac Pro 3,1 which will find use as another less important machine.

We plan on doing a business lease for the machine, I'll find out today if I'm inhibited by that in any big way. My biggest questions: Are there any known issues with the hard drives that HP has as options during the build? I am debating between a 1x256gb for the Win7 OS & Apps, then 3x300gb 15k SAS drives in a RAID 5 array for working drives. Those are all HP drives, and I am considering replacing those working raided drives with SSDs. I can have the work drives come pre-raided with an LSI MegaRAID SAS 9260-8i. That is convenient, but I could prefer an ATTO card just because I have seen them perform well over the years.

The internal HD setup and raid configuration continues to be my primary concern. We'll be doing the E5 2687 3.1ghz config with 64gb RAM (I'm on the fence and may go 128gb). We might end up purchasing one of the OWC Jupiter offerings for external fast data, once more details/pricing are known. LTO setup for backups, most likely.

Cheers and many thanks.

 

[Luke Boyce]

I've had a Z820 in my possession here for about a day now. Overall I like it a lot. HP did alright with the PCIe slot arrangement, but I've encountered one issue I'm not too happy about -- the lowest PCIe X16 slot is perfect for accepting double-width cards as the adjacent slot (the bottom one) is a standard PCI 32bit legacy slot. Unfortunately, the Quadro 6000 (and also the GTX580 or similar) won't fit there because of all the other stuff running along the bottom of the motherboard. Seems like a real goof to me. Oh well... The GTX580 or 680 HydroCopper versions from EVGA would fit like a charm, but I would have to run an external reservoir and radiator as there's no way to incorporate it into the internal system without sacrificing a drive bay. I have all the bays filled and have even mounted my system drive, the SSD, in the gap above the top 5.25" drive bay. So that gives me the SSD, the Blu-Ray writer, and 6 x 2TB HDD's for workspace.

I did a custom configuration of the Z820 with the 2687W's and Liquid cooling and it was QUITE a bit more expensive than I was expecting. Not sure it's worth the extra money to be honest, even though it is very tempting. Just don't have the budget to go that route, unfortunately.

2690's are a consideration, they seem to do just fine with air cooling. In the end, price will be similar. A good liquid system or, self-contained liquid coolers like what HP is using, come at a premium so that just about offsets the lower cost of the 2687W CPUs. Performance is going to be mostly similar, but the 2687's do score better in intensive tests and benchmarks.

I think I'll definitely still go with the 2687w's and just get a liquid cooler. Do you have any suggestions by chance? Never had to look into them, to be honest. That's pretty new ground for me. Would something like this (http://www.newegg.com/Product/Product.aspx?Item=N82E16835181017) work? I'd have to get one for each CPU, correct?

Also, I'm now wondering if I don't want to look into a different chassis than the pre-wired Supermicro one. Are there any specifics I need to keep in mind with that. I've basically just been looking around at the E-ATX compatible cases on Newegg. From what I can tell I basically need to look into something that's ready for liquid cooling, right? Would cases along these lines work: http://www.newegg.com/Product/Product.aspx?Item=N82E16811147157 (Pretty new to this, can you tell)? And of course, I have to make sure I get a power supply as well, I realize (This? http://www.newegg.com/Product/Product.aspx?Item=N82E16817139014)

I have 64GB in both the HP system and the other SuperMicro based system I've put together. At least for the moment until my other RAM arrives (today). 64GB is a good amount. But for intense rendering where I can maximize all the cores/threads on this system, it's a bit tight. I would recommend the Samsung or Micron DIMMs if you can find them, it's a pain. I had a round-about way of getting them from Micron. There's these modules here by Kingston, which should also work just fine, but they're a bit slower at CAS-11. Which is what the HP system is using. They're HP-labeled, but I'm positive these are the same modules in my Z820. http://www.newegg.com/Product/Produc...82E16820239015

That RAM doesn't look so bad in price actually. Seems I could go 128gb with those at only a few hundred dollars more.

Only card I would potentially recommend over the 580 right now would be the Quadro 6000. It's going to be slower for some stuff, especially performance visuals and pushing textures around. On the other hand, even with 3GB you can bottleneck the card with that RAM amount if you're really loading it up with a lot of data. Overall it's better tuned and runs silky smooth in various 3D apps. I like it, but it's pricey. I'll be doing comparison testing between the Quadro 6000 and GTX580 in the HP system this weekend. I still have to get all my software installed though, I just finished installing all the HDDs a few minutes ago and out of time for today on that project.

Looking forward to those 6000 vs. 580 tests. I also noticed this weekend that you can actually throw a 580 in the Mac Pro with 10.7.3 and it's plug 'n play. Do you have experience with that? Are there any caveats I should know about if I wanted to just splurge at the moment and get one to spice up my current Premiere system on my 2008 Mac Pro? I read there may be issues with the 6-pin this or that. Didn't do any extensive reading, but the possibility is intriquing.

 

[Olivier Madar]

Thank you very, very, very much Jeff,


I will go the HP route then.



Olivier

I wouldn't say the SuperMicro is better. Only real difference is the PCI slot configuration and since you're assembling yourself, you can place it in as big of a case as you would like. The HP system is very quiet. Comparable to a Mac Pro, the only fan that I audibly hear ramp up under load so far is the one on the Quadro 6000.

The SuperMicro build on my end is noisier at the moment, but I can make it quieter by swapping out fans and whatnot. Not sure what else to say on that, it's as quiet as you can build it. If you want the quietest option, you will want to go liquid cooled.

 

[Jeff Kilgroe]

I did a custom configuration of the Z820 with the 2687W's and Liquid cooling and it was QUITE a bit more expensive than I was expecting. Not sure it's worth the extra money to be honest, even though it is very tempting. Just don't have the budget to go that route, unfortunately.

My current SuperMicro based build and my HP are essentially identical in spec.. The HP worked out to about $1300 more, so yes there is a premium. All things considered, it's a small price to pay if you're short on time or don't want to chase down all the parts and test and all that, plus it has the 3 year on-site warranty included. Just be sure to use the coupon codes! Without them, you won't be able to afford buying their systems. lol

That RAM doesn't look so bad in price actually. Seems I could go 128gb with those at only a few hundred dollars more.

Yeah, not too bad. I'm probably going to order some to get my HP system up to 128GB. Although I'm going to wait for the Memorial Day holiday. HP usually has coupon codes and their official modules are currently $165 each, so not much more, and I bet I can get them for the same price or less if they have a sale.

As for the liquid cooling, I don't have any recommendations at the moment. I tried both the LGA-2011 compliant cooling blocks from CoolerMaster and ASETEK and neither one would fit properly over the CPUs (not big enough to fully cover) and the Cooler Master ones wouldn't properly mount to the sockets on the SuperMicro board. Kinda weird. I never got a response from ASETEK and CoolerMaster said they are working on a solution for the new Xeons. I'm using stock air coolers at the moment and they're working OK, but noisier than I would like and the CPUs run hotter on that build than on the HP.

Looking forward to those 6000 vs. 580 tests. I also noticed this weekend that you can actually throw a 580 in the Mac Pro with 10.7.3 and it's plug 'n play. Do you have experience with that? Are there any caveats I should know about if I wanted to just splurge at the moment and get one to spice up my current Premiere system on my 2008 Mac Pro? I read there may be issues with the 6-pin this or that. Didn't do any extensive reading, but the possibility is intriquing.

I'm looking forward to doing them here in the next couple days, just been having trouble getting to it. I'm not sure about the '08 Mac Pro with a GTX580... You're kinda reaching there and I see other bottlenecks holding you back. For starters, the RAM on the system is clocked at 800Mhz and it's already the bottleneck when working at 1080p and higher in Resolve on that platform. I would also be curious as to the power abilities in the older tower and if it really could drive a 580. It works alright in a 2009/2010 tower -- I addressed it in the Ne Mac Pro thread in this forum a little bit ago. In an '08 Mac Pro, I think I would go more for a GTX460 or even scrounge up an actual Mac Edition GTX285. It's still a great card and actually has pretty darn good CUDA performance. So does the Quadro 4000 Mac Edition. Of course the GTX580 is better if you can get it up and running, but like I said, I see other bottlenecks holding you back with the older system.

Thank you very, very, very much Jeff,


I will go the HP route then.

Cool. The quietest option is to choose their liquid cooling solution -- which is two self-contained liquid-based CPU coolers.

Be sure to track down all the HP discount codes! They have a 20% off code and depending on other factors, you can often get another 3% off of that, but it doesn't look like that one is running at the moment, expired April 30. The 20% runs through July. Which at that time, they'll probably adjust pricing and issue new coupons. HP always has big coupons like that 20% to get their system prices down into the realm of normal.

 

[Luke Boyce]

As for the liquid cooling, I don't have any recommendations at the moment. I tried both the LGA-2011 compliant cooling blocks from CoolerMaster and ASETEK and neither one would fit properly over the CPUs (not big enough to fully cover) and the Cooler Master ones wouldn't properly mount to the sockets on the SuperMicro board. Kinda weird. I never got a response from ASETEK and CoolerMaster said they are working on a solution for the new Xeons. I'm using stock air coolers at the moment and they're working OK, but noisier than I would like and the CPUs run hotter on that build than on the HP.

Hmm. Interesting. I was looking at the Corsair H80. I'll have to dig a little deeper.

I'm looking forward to doing them here in the next couple days, just been having trouble getting to it. I'm not sure about the '08 Mac Pro with a GTX580... You're kinda reaching there and I see other bottlenecks holding you back. For starters, the RAM on the system is clocked at 800Mhz and it's already the bottleneck when working at 1080p and higher in Resolve on that platform. I would also be curious as to the power abilities in the older tower and if it really could drive a 580. It works alright in a 2009/2010 tower -- I addressed it in the Ne Mac Pro thread in this forum a little bit ago. In an '08 Mac Pro, I think I would go more for a GTX460 or even scrounge up an actual Mac Edition GTX285. It's still a great card and actually has pretty darn good CUDA performance. So does the Quadro 4000 Mac Edition. Of course the GTX580 is better if you can get it up and running, but like I said, I see other bottlenecks holding you back with the older system.

Good to know. Think I'll just not worry about it at the moment. Thanks for the info.

My current SuperMicro based build and my HP are essentially identical in spec.. The HP worked out to about $1300 more, so yes there is a premium. All things considered, it's a small price to pay if you're short on time or don't want to chase down all the parts and test and all that, plus it has the 3 year on-site warranty included. Just be sure to use the coupon codes! Without them, you won't be able to afford buying their systems. lol

Hmm. I've priced my PC build at only $8k including the 128gb of RAM and the Dell monitor, but even with the 20% off I couldn't get the HP under $10k, and that was without a monitor. Although, honestly, the PC build doesn't really scare me at all. I enjoy the process of hunting down all the parts, and I've got a friend who builds servers regularly who is willing to do the actual build for me. Just hoping to make sure I get all the right components.

Also, I'm now wondering if I don't want to look into a different chassis than the pre-wired Supermicro one. Are there any specifics I need to keep in mind with that. I've basically just been looking around at the E-ATX compatible cases on Newegg. From what I can tell I basically need to look into something that's ready for liquid cooling, right? Would cases along these lines work: http://www.newegg.com/Product/Produc...82E16811147157 (Pretty new to this, can you tell)? And of course, I have to make sure I get a power supply as well, I realize (This? http://www.newegg.com/Product/Produc...82E16817139014).

 

[Jeff Kilgroe]

Your new egg links are broken. I guess I could try plugging in the product numbers maybe.

My system thru HP was around $8300 after the 20% off and after another 3% off for preferred business customers, but that promotion expired on 4/30, the 20% is still going.. So my final price was about $8700 after tax and shipping. I got the shipping credited because I paid for 2-day and they shipped it ground. I ordered the dual 2687W CPUs (liquid cooling upgrade required), 64GB (8 x 8GB - 2CPU option), and 1 x 2TB internal HDD with the Blu-Ray writer. No video card. Looking back on it, I should have loaded up the system with 2TB HDD's. They're using Hitachi UltraStars and after the discounts, the price for those HDDs would have been nearly $50 each cheaper than NewEgg. oh well...

I pulled the Hitachi drive it shipped with and installed 6 matched WD 2TB RE4 HDD's. I've got those 6HDDs in a RAID-0 now and I'm running an OCZ Vertex4 SSD as my system drive. I mounted it via its included 3.5" adapter plate and zip-tying it off in the gap above the upper drive bays. I don't know why, but HP shipped the Blu-Ray in the middle of the 3 external bays, so I swapped it to the top. Not that it matters. The Vertex4 SSD has some firmware issues, which OCZ says they're aware of. But after disabling the option to put it to sleep the drives and disabling hybrid sleep, the SSD performs OK. I do get errors in the system logs about the SSD taking too long to wake from sleep. Which it does take a while. Hybrid sleep on the system causes a blue-screen / reboot almost every other time the system wakes, due to the SSD slow wake-up issue.

So far, I've encountered 3 "annoyances" with the HP system: 1> You can't mount anything in the spare drive bays, I had to go back and order two additional drive sleds at $40 each. 2> The second SATA header that consists of 4 channels at 3Gbps, is not bootable. Can only boot from the dual-port 6Gbps SATA header or the 8-port LSI SAS/SATA host. 3> The bottom X16 PCIe 3.0 slot would be perfect for double-width GPUs and other cards as the adjacent slot space at the very bottom of the system is a legacy PCI connector. Unfortunately, it's near impossible to wedge a large GPU card in there because of the amount of cables / connectors along the bottom of the motherboard there. Qualifies as a design flaw, IMO. On the other hand, if I decide to ditch the Quadro card (and I might if the GeForce performance is where I want it), then I'll go to the EVGA HydroCopper GPUs. I can then fit one in this lowest X16 slot and run the water lines out the slot space of the bottom legacy slot to an external radiator and reservoir. ...Or maybe that's too much of a pain. Not sure I want to get into external water systems and whatnot with this system. As of right now, it performs great with the Quadro 6000 card. Ill have to try other GPU combinations and see what the best way is for Resolve and other apps I'm running.

 

[Mark Orion]

I have a question I'd sure appreciate your opinion on. I have a 2008 Mac Pro 8 Core 2.8 kHz with 10GB of RAM.

Since I'm waiting to see when new Mac Pros arrive, would I get a noticeable performance improvement in CS6 if I upgrade to 32GB of RAM?

Thank you!

 

[Jeff Kilgroe]

Mark, upgrading to 32GB would help, but probably not in a huge way. If you're careful about not holding too many apps open at the same time, you should be able to squeak by just fine with what you have. That would be an expensive RAM upgrade and money probably better spent elsewhere. Do you intend to hold onto the '08 Mac for much longer?

 

[Jeff Kilgroe]

Just an FYI for those of you who have been following my posts regarding the dual Xeon E5 systems and HP Z820... The EVGA GTX580 Classified Ultra *WILL NOT* physically fit into the HP Z820. If I were to leave the PCI e bracket retainer loose and secure the cards with screws, then leave the PCIe airflow cover out of the mix, I can get the lid on the system. Not going to go that route.

Other than that, the GTX580 tests are looking good! GPU tests are progressing nicely and it looks like I may be sticking with dual GTX580 cards in my SuperMicro build and sticking with the Quadro 6000 in my Z820. I need a second card in the Z820 for additional CUDA acceleration. I'm torn between a lower clocked GTX580 (which will fit) or going with a Tesla C2075 which makes the HP into a "Maximus" configuration. Wish I had the Tesla here to benchmark with. I've heard it's pretty snappy, even though the specs on paper don't seem to show that.

Will have my final reports soon...

 

[Mark Orion]

Mark, upgrading to 32GB would help, but probably not in a huge way. If you're careful about not holding too many apps open at the same time, you should be able to squeak by just fine with what you have. That would be an expensive RAM upgrade and money probably better spent elsewhere. Do you intend to hold onto the '08 Mac for much longer?

Jeff, thanks so much for your insight. I'll probably hold on to the '08 Mac and wait to put my money into something that would provide a significant performance increase.

 

[Jose Lomeña]

Mark, I have the last 2x2.93 macpro. I bought it with 12gb, and when I updated to 48gb i saw an increase of 2x in speed with after effects render. Maybe it can be this specific ae project... maybe not. But 2x speed increase is a lot. With this specific project cpu only top at 500%. I think 10GB is too low for ae in some projects with 4k images. I think you need 12gb only for ae. I recommend to update at least to 16gb, maybe it didn't cost so much and after effects will have near double of free mem. (Maybe you can extract 2x1gb modules and add 2x4gb modules = 300$ with owc mem, i didn't test it)

 

[Jeff Kilgroe]

Mark, what is your current memory configuration now? The '08 Mac Pro has a dual-channel memory arrangement and juggling around memory modules or exchanging various modules could give you a significant speed increase. Going all the way to 32GB via 4GB modules of that kind doesn't make a whole lot of sense, IMO. It's going to run $950-$1100 to do it and the actual value of the '08 Mac Pro at this point is pretty low. Such is the way technological cookies crumble... Moving to 16GB via 8x2GB might be an OK compromise. You will get the fastest memory performance at least and generally speaking you shouldn't run out of RAM if you're conscious about which apps you have up and running. As for AE, it's a memory hog now that it's 64bit and if you're working with comps at 1080p and up with a lot of layers, you can eat up RAM pretty fast.

32GB is in a way a bit too much for the '08 Mac Pro or systems of that generation and prior. Sure, having it is great, but there are so many other performance restrictions and bottlenecks compared to newer hardware, it just seems questionable as to whether one should really dump more money into a 4 year old system.

Another option is to see if there's a used RAM kit out there you can pick up for cheap.

 

[Jason Enzer]

jeff,

do you have a dual z820? if so have you tested the memory bandwidth? i have been seeing less than desirable memory write benchmarks. in fact every dual processor c602 chipset board i have tested yields sub 11GB a sec memory write speed. even intel is looking into the issue as they are getting reports from numerous customers.

can you try maxxmem2 to assess if you havent already?

thx!

-jason

Just an FYI for those of you who have been following my posts regarding the dual Xeon E5 systems and HP Z820... The EVGA GTX580 Classified Ultra *WILL NOT* physically fit into the HP Z820. If I were to leave the PCI e bracket retainer loose and secure the cards with screws, then leave the PCIe airflow cover out of the mix, I can get the lid on the system. Not going to go that route.

Other than that, the GTX580 tests are looking good! GPU tests are progressing nicely and it looks like I may be sticking with dual GTX580 cards in my SuperMicro build and sticking with the Quadro 6000 in my Z820. I need a second card in the Z820 for additional CUDA acceleration. I'm torn between a lower clocked GTX580 (which will fit) or going with a Tesla C2075 which makes the HP into a "Maximus" configuration. Wish I had the Tesla here to benchmark with. I've heard it's pretty snappy, even though the specs on paper don't seem to show that.

Will have my final reports soon...

 

[Mark Orion]

Mark, what is your current memory configuration now? The '08 Mac Pro has a dual-channel memory arrangement and juggling around memory modules or exchanging various modules could give you a significant speed increase. Going all the way to 32GB via 4GB modules of that kind doesn't make a whole lot of sense, IMO. It's going to run $950-$1100 to do it and the actual value of the '08 Mac Pro at this point is pretty low. Such is the way technological cookies crumble... Moving to 16GB via 8x2GB might be an OK compromise. You will get the fastest memory performance at least and generally speaking you shouldn't run out of RAM if you're conscious about which apps you have up and running. As for AE, it's a memory hog now that it's 64bit and if you're working with comps at 1080p and up with a lot of layers, you can eat up RAM pretty fast.

32GB is in a way a bit too much for the '08 Mac Pro or systems of that generation and prior. Sure, having it is great, but there are so many other performance restrictions and bottlenecks compared to newer hardware, it just seems questionable as to whether one should really dump more money into a 4 year old system.

Another option is to see if there's a used RAM kit out there you can pick up for cheap.

Jeff, Here is the current configuration:

DIMM Riser B/DIMM 1 2GB
DIMM Riser B/DIMM 2 2GB
DIMM Riser A/DIMM 1 1GB
DIMM Riser A/DIMM 2 1GB
DIMM Riser B/DIMM 3 empty
DIMM Riser B/DIMM 4 empty
DIMM Riser A/DIMM 3 2GB
DIMM Riser A/DIMM 4 2GB

Looks like 4 more 2GB might be a good bang for the buck - thanks!

Thanks!